Safe place to store user-defined parameters

slbaumgartner

Guys,

Please forgive an old fart for a brief history lesson.

The tmp, temp, and similar directories/folders came into being as a means for a program to create a working file during a run without concern for the physical location of the file. The intent was that the file would no longer be needed after the program exits and could be cleaned up by the OS at any time after the program closes. The fact that an OS may have a lazy cleanup policy that lets such files hang around for a long time is not a good excuse for ignoring the basic idea that these files are meant to be temporary! These are not reliable places to store persistent parameters. The policy could change at any time!

Steve

fredo6

@slbaumgartner said:

Guys,

Please forgive an old fart for a brief history lesson.

The tmp, temp, and similar directories/folders came into being as a means for a program to create a working file during a run without concern for the physical location of the file. The intent was that the file would no longer be needed after the program exits and could be cleaned up by the OS at any time after the program closes. The fact that an OS may have a lazy cleanup policy that lets such files hang around for a long time is not a good excuse for ignoring the basic idea that these files are meant to be temporary! These are not reliable places to store persistent parameters. The policy could change at any time!

Steve

I understand.
But where can the persistent parameters be saved on Mac?
What are other applications doing?

Fred

driven

Apple applications must be self contained to meet the 'Approved' status that SU is attempting to gain.

If they don't 90% of mac users will get a warning that 'SU' is 'unapproved' and 'potentially harmful', this puts a LOT of people off.

That's why everything is moving back into SU's App Folder in the User space it's also why they added Extension Manager.

As far as Apple are concerned Users should only ever need to access an app from within the app and should not be required to ever need to add anything externally.

For it's Users. SU provides the all the paths you need to supply an SU Plugin to run on a mac, use them.

If Users have move stuff around, that's their problem and SU and Apple are happy.

If your developing on a mac, you can change permissions to get around some of the sandboxing and you can then compile and 'package' helpers so Uses can benefit from your workarounds, if your on a PC it's a nightmare to take alternate routes.

If you keep all your files in your Plugins Directory and the SU at supplied external paths, I can't see a problem.

SU and Apple are happy, you can be happy too.

john

Aerilius

That's very easy, if the OS vendor finger-points at the one who plays bad, then everyone has to try to play nice.
User's personal data (settings on one side and user data (works, templates, user-managed plugins on the other side) belong to the user level also on other operating systems (of which Windows is only one special example, I even do not use Windows). Then there would not be permission problems. It has often enough been pointed out that plugins in the Windows version SketchUp are in the wrong place.

The permission problem story on Windows has its origin that it was designed as a single user OS at a time when also security models where basic or non-existing. Microsoft decided to keep compatibility instead of "deprecating" older applications (or applications that don't follow the new rules), which lead to this inconsistency.

slbaumgartner

@fredo6 said:

@slbaumgartner said:

Guys,

Please forgive an old fart for a brief history lesson.

The tmp, temp, and similar directories/folders came into being as a means for a program to create a working file during a run without concern for the physical location of the file. The intent was that the file would no longer be needed after the program exits and could be cleaned up by the OS at any time after the program closes. The fact that an OS may have a lazy cleanup policy that lets such files hang around for a long time is not a good excuse for ignoring the basic idea that these files are meant to be temporary! These are not reliable places to store persistent parameters. The policy could change at any time!

Steve

I understand.
But where can the persistent parameters be saved on Mac?
What are other applications doing?

Fred

On a Mac under OS X, the conventional/approved place for user configuration parameters is a .plist file in ~/Library/Preferences. The file name starts with a dot-separated sequence like a reversed domain name that identifies the relevant app, and then adds more segments to specify what about that app. For example, com.trimble.sketchup.fredo.magictool.plist.

There are API calls for storing and retrieving info from plist files, but I don't think they are exposed in the SU Ruby API, so using them would probably require an external helper program or added Ruby file to get at plists in a clean way (Apple discourages reading and writing the files manually). I have no experience with them, but there are some plist generator and parser Ruby files on source forge.

driven

@slbaumgartner said:

but I don't think they are exposed in the SU Ruby API, so using them would probably require an external helper program or added Ruby file to get at plists in a clean way (Apple discourages reading and writing the files manually). I have no experience with them, but there are some plist generator and parser Ruby files on source forge.

@Steve,
that's exactly what this pair does, your entry in the com.sketchup.SketchUp.plist

 result = Sketchup.write_default "section",
   "variable", "my_value"
result = Sketchup.read_default "section",
 "variable", "default"

in the .plist that shows as

	<dict>
		<key>variable</key>
		<string>"my_value"</string>
	</dict>

john

slbaumgartner

@driven said:

@slbaumgartner said:

but I don't think they are exposed in the SU Ruby API, so using them would probably require an external helper program or added Ruby file to get at plists in a clean way (Apple discourages reading and writing the files manually). I have no experience with them, but there are some plist generator and parser Ruby files on source forge.

@Steve,
that's exactly what this pair does, your entry in the com.sketchup.SketchUp.plist

 result = Sketchup.write_default "section",
>    "variable", "my_value"
> result = Sketchup.read_default "section",
>  "variable", "default"
>

in the .plist that shows as

	<dict>
> 		<key>variable</key>
> 		<string>"my_value"</string>
> 	</dict>

john

Which leaves me somewhat confused about what fredo is seeking to accomplish. Unless you are determined to keep your info somewhere separate from SketchUp's "defaults", what's the problem with using these methods?

Steve

Aerilius

I think for simple configurations (keys, values etc.) Sketchup.write_default is the ideal solution because it uses the API, and the API makers care about that it works.
But sometimes one wants to store more data or even files, so called "user data" (vs. "config").
I don't know what Fredo has in mind.

fredo6

Aerilius is right.

There are situation where you can persist parameters in the Sketchup section of the registry and some where this is not desirable or not possible.

My scripts use the registry for some parameters, and files for some others. By principle, it is never advised to overload the registry of a system, as it may easily get corrupted.

In addition, there are cases where you simply need to store files, not data, say components, materials, etc... that are created on the fly by the plugin based on user inputs.

Windows makes provision for a 'application data' location on disk. On MAC, there must also be a 'natural' place where such user files are stored. For instance, Microsoft Office use templates and other option files and they are not stored in the registry.

I think Sketchup cannot continue to propose only the Sketchup root directory to store files underneath. All new versions of operating systems are very picky on security and won't let you access this area from a program or embedded script like Ruby.

The objective of this post was to see whether the developer community would have ideas and would agree on a convention, because I assume many scripts are in the same situation.

Fredo

Chris Fullmer

I'm recently beginning to learn Mac - would:

~/Library/Application Support/

be the folder that is comparable to Application Data on Win?

driven

hi
have a look at this, it explains Apples viewpoint.

http://developer.apple.com/library/ios/#documentation/FileManagement/Conceptual/FileSystemProgrammingGUide/FileSystemOverview/FileSystemOverview.html

Apples basic philosophy is
@unknownuser said:

Every app is an island

this includes any plugins written for any app.

Apple are moving away from any need for "uninstaller software", as far as they're concerned if a User or the "App Store", trashes or updates an app, then 'all' the files associated with the outgoing 'app version' will be gone on next restart of the mac. To achieve this, it's limiting 'where' you can easily write to, and for 'app approval' you have to prove this can be achieved.

If I remove anyones 'Plugin' for SU, I expect 'all' it's support files to disappear from my system. So does Apple.

I have one 'Plugin' that moves items all over the place, but if I remove it or even just disable it from the 'Extension Manager' it puts everything back in place and deletes it's 'watch-file'. If I then restart it it writes everything back again. The only thing that ever resides external to SU folder structure are temporary or the outcome of User input i.e. document files I want to keep.

john

Chris Fullmer

So John, what is the simple answer to Fredo's question? How does he safely write files to the HD on a Mac?
if Chrome downloads a plugin, it has to put it somewhere. Does it put it in the app bundle?

driven

@Chris

for SU the is most comparable to
@unknownuser said:

Application Data on Win
is ~/Library/Application Support/SketchUp 2013/SketchUp

if I add a folder in there...
Sketchup.find_support_file("SketchUp", "john") SHOULD work, but it doesn't.

john
PS:
we cross posted so I'll add 2 answers
1.
@unknownuser said:

Google Chrome installs the extension into a randomly generated directory in ~/Library/Application Support/Google/Chrome/Default/Extensions and registers it in its Preferences file.

The User "Plugins" folder unless it needs to work when plugins are disabled. [really only for debugging or maybe a timekeeping tool]

driven

@Fredo

I feel your pain, love your plugins, maybe someone at Trimble or Steve can tidy this up better then me.

it writes a binary .plist in Users Plugins Folder, in your plugins subdirectory, you can then access it as you would the SU one, but it's your own.
Once written, it can be read/write i.e. doesn't need a restart [unlike SU's which does]
you can store all types of thing in it include binary images etc...

#!/usr/bin/env ruby
# DON'T RUN DIRECTLY FROM SKETCHUP SEE USAGE BELOW
# put this in your folder and call it to create a binary .plist on the Users mac
# write what you will
require "osx/cocoa"
require "pp"
# Create the plist and assign values
my_path = File.expand_path("~/Library/Application Support/SketchUp 2013/SketchUp/Plugins/FredoHome/__me_first/com.fredo.example.plist")
myfile = my_path
my_dict = OSX;;NSMutableDictionary.dictionary
my_dict['The_Color'] = 'blue'
my_dict['count'] = 15
#
# Output to File and print
my_dict.writeToFile_atomically(myfile, true)
pp my_dict

=begin
### Usage

make_plist = %x(exec ~"/Library/Application\ Support/SketchUp\ 2013/SketchUp/Plugins/FredoHome/__me_first/mac_plist.rb" 2>&1)
my_plist = File.expand_path("~/Library/Application Support/SketchUp 2013/SketchUp/Plugins/FredoHome/__me_first/com.fredo.example")

add2my_plist = %x(defaults write "#{my_plist}" animation-duration -float 0.12)

inQuire = %x(defaults read "#{my_plist}")
{
    "The_Color" = blue;
    "animation-duration" = "0.12";
    count = 15;
}

inQuire = %x(defaults read "#{my_plist}"  The_Color)
blue
=end

the format of this bit is important com.fredo.example.plist for defaults to read and write.
Lifted from the man page... https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/defaults.1.html

     Specifying value types for preference keys;

                 If no type flag is provided, defaults will assume the value is a string. For best results,
                 use one of the type flags, listed below.

     -string     Allows the user to specify a string as the value for the given preference key.

     -data       Allows the user to specify a bunch of raw data bytes as the value for the given preference
                 key.  The data must be provided in hexidecimal.

     -int[eger]  Allows the user to specify an integer as the value for the given preference key.

     -float      Allows the user to specify a floating point number as the value for the given preference
                 key.

     -bool[ean]  Allows the user to specify a boolean as the value for the given preference key.  Value must
                 be TRUE, FALSE, YES, or NO.

     -date       Allows the user to specify a date as the value for the given preference key.

     -array      Allows the user to specify an array as the value for the given preference key;

                       defaults write somedomain preferenceKey -array element1 element2 element3

                 The specified array overwrites the value of the key if the key was present at the time of
                 the write. If the key was not present, it is created with the new value.

     -array-add  Allows the user to add new elements to the end of an array for a key which has an array as
                 its value. Usage is the same as -array above. If the key was not present, it is created
                 with the specified array as its value.

     -dict       Allows the user to add a dictionary to the defaults database for a domain.  Keys and values
                 are specified in order;

                       defaults write somedomain preferenceKey -dict key1 value1 key2 value2

                 The specified dictionary overwrites the value of the key if the key was present at the time
                 of the write. If the key was not present, it is created with the new value.

     -dict-add   Allows the user to add new key/value pairs to a dictionary for a key which has a dictionary
                 as its value. Usage is the same as -dict above. If the key was not present, it is created
                 with the specified dictionary as its value.

     Specifying a host for preferences;

     Operations on the defaults database normally apply to any host the user may log in on, but may be
     restricted to apply only to a specific host.

               If no host is provided, preferences operations will apply to any host the user may log in on.

     -currentHost
               Restricts preferences operations to the host the user is currently logged in on.

     -host hostname
               Restricts preferences operations to hostname.

Chris Fullmer

Talking it over a little bit here with a Mac person, I still think that the ~/Library/Application Support/ folder is the right starting point. It exists on all Macs, and is specifically designed to be the location that support files get saved to. To me, that is the definition of what Fredo is looking for, and it is sanctioned by Mac. So it CAN'T be against their own "Every app is an island" viewpoint, or else they wouldn't have created it, right?

Chris Fullmer

Ok, I'm reading through that great Link John (very helpful to me as I try to wrap my head around Macs). 2/3 of the way down, in Table 1.3 it does explain the purpose of the Application Support folder:

@unknownuser said:

Use this directory to store all app data files except those associated with the user’s documents. For example, you might use this directory to store app-created data files, configuration files, templates, or other fixed or modifiable resources that are managed by the app. An app might use this directory to store a modifiable copy of resources contained initially in the app’s bundle. A game might use this directory to store new levels purchased by the user and downloaded from a server.
All content in this directory should be placed in a custom subdirectory whose name is that of your app’s bundle identifier or your company.

driven

I'll wait till your finished before I bite your head off...

slbaumgartner

@chris fullmer said:

Talking it over a little bit here with a Mac person, I still think that the ~/Library/Application Support/ folder is the right starting point. It exists on all Macs, and is specifically designed to be the location that support files get saved to. To me, that is the definition of what Fredo is looking for, and it is sanctioned by Mac. So it CAN'T be against their own "Every app is an island" viewpoint, or else they wouldn't have created it, right?

On iOS, apps are quite strictly sandboxed. Each app gets its own virtual disk image and can't read or write anything outside that image.

But in Mac OS X, this is not yet the case. An app can read and write anything for which the user has adequate permission anywhere on any drive. And, using shell scripting such as John has provided, a plugin can ultimately bypass the app's own API and rules to execute arbitrary UNIX commands.

The sandboxing rules today on OS X only exist as conventions that developers are asked to respect (required to respect if their code is published via the App Store). But, as John has alluded, Apple is trying to beef up the sandboxing because so many apps spew files all over the disk and this makes it difficult or impossible to completely remove an app. Rumor is that the next edition ("Mavericks", expected in the Fall) will have some changes in this regard.

So, whereas ~/Library/Application Support/<appnane>/... has been the conventional place for applications to save helper files, this could change.

Based on all of this, it seems to me that the only place that simultaneously provides:

assured read and write permission for the user
logical/code consistency between Windows and Mac
support within SketchUp for readily finding the folder by a plugin

is a plugin's own subfolder within whatever folder SketchUp is using to store user-added extensions. That is, when you tell SU to add an extension, SU itself chooses the appropriate folder ("Plugins" somewhere) and unzips your rbz file there. You have to have write permission in that folder, else SU couldn't install the extension. At that point your plugin code can query to find out where it was installed and can create more subfolders as well as write and read arbitrary files there. The plugin should not hard code any path such as "C:\Program Files\SketchUp\Plugins" or "~/Library/Application Support/SketchUp 2013/SketchUp/Plugins". It should use either Sketchup.find_support_file(...) or File.dirname(FILE) or something similar to get SU to reveal where the extension was loaded, and then use relative paths from there.

That approach both allows arbitrary files to be written and read, but also will survive if SU is compelled to change the physical folder where extensions are stored because OS security rules were tightened.

Steve

Chris Fullmer

@driven said:

I'll wait till your finished before I bite your head off...

I'm done John, you can start chewing

Chris Fullmer

@Steve - Writing to a plugin's folder on Windows can fail though, so its not really a god solution for Windows. And up until 2013, it should have been equally likely to fail on Mac as well, because plugins were being written to /Library/Application Support, which is public, which requires admin rights. Only since 2013 have Plugins been written to ~/Library/Application Support/..., which is completely writeable by the current user.

Also, we're not writing in iOS, so their extreme sandboxing isn't a concern here (yet?). It will be interesting to see what Maverick implements. But its hard enough to develop for backwards compatibility and current compatibility....forward speculative compatibility might be a little too speculative for my taste.

Chris

PS - for the record, I have nothing to do with Macs and the decisions we make about how we handle the Mac files system, so don't let me ignorance scare you too much. They won't let me ruin SU on your Mac any time soon