Safe place to store user-defined parameters

slbaumgartner

@tig said:

This discussion seems to go on and on and ignore the simplest of solutions.
Find the users 'Temp' folder [names vary with OS etc]...
I have previously outlined how to get this for MAC and PC.
This 'Temp' folder is useful for holding temporary data, it can be purged by the system, so permament data needs to reside elsewhere...
This 'Local/Temp' folder is in a 'folder tree'.
The folder containing the 'Temp' folder can be found using File.dirname(tempfolderpath).
That container folder can be used to make your own subfolder to hold your more permanent data.
That container folder has the same unrestricted rights for the User as the Temp folder itself.
It is intended to hold app-specific folders/data...
Just open the C:/Users/User/Local folder to see this...

What is the problem in doing it this way?
I have done it thus for a long time, without any issues whatsoever...

TIG - already answered. All of the tmp, temp, etc folders are meant for working files used while a program is running. No OS that I know of promises to keep such files around after the program exits. It may have been working for you, but that is only because the OS is lazy about cleaning up. This could change with any update. Also, most "disk cleanup" programs will erase the contents of these folders, so you are running risk of a user blowing away your data and then blaming you.

Steve

Aerilius

TIG speaks not of the temporary folder itself, but of its "parent folder". Under the assumptions of Windows users this would be a writable app data directory (exactly what we've been searching): C:\Users\<username>\AppData\Local%(#999999)[\Temp] (Probably equal to what we get from the APPDATA environment variable.)
But we cannot blindly take a parent folder without knowing what it is. There is nothing wrong if other operating systems have the temporary folder at root level /tmp with no parent. Still not a single solution without OS-specific switches.

TIG

On a modern PC the user's Temp folder's 'container' is going to be C:/Users/User/AppData/Local/: or something slightly different on XP...
On a MAC it's a seemingly meaningless folder name that contains the user's temp folder [named ' T']: something like /var/folders/rp/b9k42l5x7xngx_8tckgs0zdr0000gn/
Both of these 'containers' are consistently made and are intended to be always be fully read/writable by the user and the user's applications - that is why they are there.
The Unix-like /tmp folder should never be needed, and it's not returned by ENV [my earlier illustrations of a hierarchy of temp folders had it in as a 'fall back' should all else failed]...

Other apps use the 'Local' folder to store their data in their own subfolders.
The differences between Local and Roaming were also spelled out in one of my earlier posts in this thread.

slbaumgartner

@tig said:

On a modern PC the user's Temp folder's 'container' is going to be C:/Users/User/AppData/Local/: or something slightly different on XP...
On a MAC it's a seemingly meaningless folder name that contains the user's temp folder [named ' T']: something like /var/folders/rp/b9k42l5x7xngx_8tckgs0zdr0000gn/
Both of these 'containers' are consistently made and are intended to be always be fully read/writable by the user and the user's applications - that is why they are there.
The Unix-like /tmp folder should never be needed, and it's not returned by ENV [my earlier illustrations of a hierarchy of temp folders had it in as a 'fall back' should all else failed]...

Other apps use the 'Local' folder to store their data in their own subfolders.
The differences between Local and Roaming were also spelled out in one of my earlier posts in this thread.

OK, so your notion is to find the user's temp folder and backtrack to its parent on the assumption that the parent is a stable, writable location.

I think you are half right: that folder will be writable. But the other half is why you are consternated by the weird path name synthesized by Mac OS X and also why this attack will fail.

Mac OS X uses its underlying UNIX library's temporary folder library, which was designed for the purposes I described. The odd folder name is a globally unique id that assures the temp files for this run of this app for this user won't collide with any other user or app's temp files. The paradigm is that an app requests a temporary file from the OS when it needs working space, gets a file handle in return, uses that file until completed, then closes the file. The app never needs to look at the full path to the temp file and has no assurance from the OS that the same folder will be used on another run or for a second temp file (after all, what if you start two instances of the same app at the same time? Do you want their working files to stomp on each other?). So, the problems arise from wanting to store persistent data using a facility that was designed for temporary data.

As already noted, Mac OS X expects that applications will put individual users' configuration data into "/Users/<userid>/Library/Application Support/<appname>/<subfolders as desired>", which is stable and is always writable by the userid. This is the nearest equivalent to Windows "C:\Users\User\AppData\Local<appname>". I don't know: is this folder entangled with magic redirection such as ThomThom described? On Mac OS X, ~/Library is hidden by default, but that's a Finder setting that has no effect on read/write access.

Steve

TIG

I didn't say I was disconcerted by the MAC's weird folder naming for the container of 'T'.
It works just fine, and you can make another subfolder in that container.
I was simply saying that is name is hardly 'user-friendly'...

I'm NOT supposing we hard-code any paths.

Using ENV each time will return the 'Temp' folder path for any OS, one way or another - see my examples earlier...
This is folder fine for storing things for that session [they might not be there next week]...

From that 'tree' we can get the 'Temp' folder's container.
That container folder is also readily writable.
It is not going to change session to session...
So if we want to store something longer than this session, then it's best to write it into your own subfolder at this level.

I see no reason to make other convoluted attempts to find another folder to write into ?
No one has yet explained why my simple approach should not be used

driven

Probably not the last word on the subject...

I ran some overnight file tools on all of the plugins folders content

I've gone through it to look for any differences of any sort

Some of fredo's had this

00000000 62 70 6C 69 73 74 30 30 55 74 65 73 74 33 08 00 |bplist00Utest3..|

it's possibly a Windows Registry Editor key?

so then I went hunting

the only files with this flag are in DEFPARAM_Dir and Cadfather png files

I did A complete clean instal checking for this flag

it doesn't reappear on any new fredo files

it appears at some point I installed some CadFather plugin that included a DEFPARAM_Dir and this flag has been moved into all my old Plugins folders.

with my fresh instal, system made a new DEFPARAM_Dir and it's clear of this or any other flag.

I believe on some macs this may be seen as a corrupt folder

another observation is that plugins downloaded and installed retain

/Downloads/FREDOTOOLS_Dir_12: com.apple.quarantine: 0002;51c32928;Safari.app;CEF9108C-32D9-4A38-91B1-51C844D26E03

but the same plugin from a SCF Store in SU instal is clear of this flag.

It's possible some macs won't run files with this flag depending on local settings.

@Fredo I think your Users may have a corrupt Folder issue.

john

driven

Permissions for the new code generated one in User Library

upstairs;~ johns_iMac$ ls -lr@   /Users/johns_iMac/Library/Application\ Support/SketchUp\ 2013/SketchUp/Plugins/DEFPARAM_Dir
total 8
-rw-r--r--  1 johns_iMac  staff  1168 Jun 20 17;05 LibFredo6.def

Permissions for the old one in System Library


upstairs;~ johns_iMac$ ls -lr@  /Library/Application\ Support/Google\ SketchUp\ 7/SketchUp/PluginsParking/DEFPARAM_Dir
total 40
-rwxr-xr-x  1 johns_iMac  admin   607 Apr  8  2009 readme.txt
-rw-r--r--  1 johns_iMac  admin  3102 Feb  2  2010 RoundCorner.def
-rw-r--r--  1 johns_iMac  admin   827 Jan 16  2010 HoverSelect.def
-rw-r--r--  1 johns_iMac  admin  3219 May 12  2009 FreeScale.def
-rw-r--r--  1 johns_iMac  admin  3164 Mar 11  2010 FredoScale.def

fredo6

@driven said:

@Fredo I think your Users may have a corrupt Folder issue.

john

John,

I am not sure I understand what you mean.
What is sure is that all my files come straight from a Windows PC (I don't have a Mac). And the Def files are generated by the scripts not coming from installation.
Corruption is always possible afterward as for any read/write operations when programs sometimes crashes.

Fredo

fredo6

@tig said:

I see no reason to make other convoluted attempts to find another folder to write into ?
No one has yet explained why my simple approach should not be used

TIG,

Temporary folder is fine for session files and fortunately the location can be safely determined on both Mac and PC via ENV variable.

For persistent files, I mean, data users need to keep across SU sessions, and even across SU versions (and plugin versions), we need a safe place which is not normally cleaned up by the system like a Temp folder would be.

On Windows, ENV["LOCALAPPDATA"] seems OK (except maybe on Windows XP according to Dan)
On Mac, it seems that /Library/Application Support/SketchUp can be and acceptable root and is writable with no security constraint.

I don't say that the problem is solved because

there was a discussion about using /Library or ~/Library
I am not too sure on how to get the root directory via a ruby mecahnism, either ENV or Sketchup.find_support_files, especially a stable method across SU6, SU7, SU8 and SU13.
As you say, it is never good to hardcode a directory path

Fredo

Aerilius

What speaks against using an iterative search?


# Find an existing and writable directory where to store user data.
dir = [
  ENV["LOCALAPPDATA"],                    # Windows ver. 6+
  ENV["APPDATA"],                         # Windows
  File.join(ENV["HOME"].to_s, ".local", "share"), # Free desktop standard
  File.join(ENV["HOME"].to_s, "Library", "Application Support"), # OS X
  "."                                     # Fallback; exists always, where as ENVs may not.
].compact.map{ |path| File.expand_path(path) } # expands links and path separators
 .find{ |path| File.exists?(path) && File.writable?(path) }
DATA_DIR = File.join(dir, "MyPlugin")

driven

@fredo6 said:

I am not sure I understand what you mean.

After one of the comments, yesterday I finally realised that the DEFPARAM_Dir was not part of the download.

I have always had one and didn't realise that you had a script that writes it, it had never been replaced because it sits outside of your other Plugins folders.

When I checked my local DEFPARAM_Dir folder it had the PC flag on it, so I had always assumed you had supplied it with those flags in your download.

Realising now that you hadn't, I looked for that flag on other folders and files and only found it on the CadFather png's and a couple of your .def files.

My conclusion is that I must have downloaded the flaged DEFPARAM_Dir, the flaged .def's and the flaged .png's with the troublesome CadFather 'SCF Toolbar' years ago.

Although I had 'purged' the CadFather 'SCF Toolbar', I must have kept the DEFPARAM_Dir from it.

Other mac users have probably done the same thing. Meaning they would also have this 'corrupt' folder that wasn't written by their mac, and may not be writable by them. From a non-admin account I could not write to that Folder or the files in it.

The best course of action for anyone having trouble with an existing DEFPARAM_Dir, is to bin it and NOT make one manually, but let your script write a new one when it sees it's missing. That way it will have all the correct permissions for it's location.

It is re-written as soon as I open SU and use 'Change Parameters' , so it's doing everything you want and need.

It is best if this is done in the User Library even on early versions of SU, your plugins work from there because they are self contained...

john

@Aerillus, I get back to yours in a moment...

Aerilius

As for the flags, I think they are an OS X feature and created after extracting the zip file (zip files have only a very primitive internal file system).

driven

@aerilius said:

As for the flags, I think they are an OS X feature and created after extracting the zip file (zip files have only a very primitive internal file system).

My point is they are from a download and DEFPARAM_Dir should have been made locally at some point in time.

In looking it up flags are common on Windows as well...

john

driven

@aerilius said:

when OS X is detected:

check if plugin directory is writable and use it

SU2013 in ~/Library/Application Support/SketchUp 2013/SketchUp…

in earlier versions only writable if owned by the user /Library/Application Support/SketchUp 8/…

otherwise do what? (question 1, to be determined),
maybe a subfolder of File.expand_path("~/Library/Application Support/SketchUp"), did this location change with the transition to Trimble?
otherwise (when platform detection fails)

check if plugin directory is writable and use it

otherwise fallback (temporary folder, user folder, ignoring or error handling)

Another question 2 is whether to put the plugins folder at highest priority if it is writable? This would make it simpler and more consistent, it would be prepared for when SketchUp moves plugins into user level on Windows, and makes plugins "self contained".

my vote is to use File.dirname(__FILE__) and TEST IT?


  my_unkown_path =  File.dirname(__FILE__) # for testing I used "/System/Library" # and  "/tmp" #

if  not (my_unkown_path[/User/])

  my_unkown_path_test = (File.join(my_unkown_path, "/.My_path_test.txt"))

begin

  File.open(my_unkown_path_test, 'w+') {|f| f.write("did you read the manual or just are you just lucky?")}

rescue  => e

  UI.messagebox("RTFM then move me #{e} ")

  abort("force error to end")

end

  my_warning = UI.messagebox "need to check you have the RIGHTS to write?"

if FileTest.readable_real?(my_unkown_path_test)

  my_good_news = UI.messagebox(IO.readlines(my_unkown_path_test))

  File.delete(my_unkown_path_test)

else

  my_bad_news = UI.messagebox("I can only guess why you might possibly think that my plugin would run from #{my_unkown_path}\n
   RTFM")

end

end

EDIT: added a rescue to test against /System/Library which I know I can't write to...
tested and works in v8 and v2013

Aerilius

If I understood right, /Library is at system level and the user level (in the SketchUp island ) is preferred in the longterm.
Not sure why you want the root, but Ruby is so nice and resolves File.expand_path("/") to root on OSX / and Windows C:\ (or whatever drive letter is root).

So maybe we can crystallize the following choices (?):
when Windows is detected:

check for ENV["LOCALAPPDATA"] and use it if available # Vista, 7, 8
otherwise check for ENV["APPDATA"] # XP
otherwise if both envs are empty, fallback to something (temporary folder, user folder, ignoring or error handling)
when OS X is detected:
check if plugin directory File.dirname(__FILE__) is writable and use it

SU2013 in ~/Library/Application Support/SketchUp 2013/SketchUp…

in earlier versions only writable if owned by the user /Library/Application Support/SketchUp 8/…

otherwise do what? (question 1, to be determined),
maybe a subfolder of File.expand_path("~/Library/Application Support/SketchUp"), did this location change with the transition to Trimble?
otherwise (when platform detection fails)
check if plugin directory is writable and use it
otherwise fallback (temporary folder, user folder, ignoring or error handling)

Another question 2 is whether to put the plugins folder at highest priority if it is writable? This would make it simpler and more consistent, it would be prepared for when SketchUp moves plugins into user level on Windows, and makes plugins "self contained".

danielbowring

Come to think of it, this could go well as part of SketchupExtension

Something like (for example only):


APPDATA_DIRECTORY = get_appdata_directory_somehow()
RESTRICTED_CHARACTERS = Regexp.new('[\\/;*?"<>|%]')

class SketchupExtension
    def data_directory()
        dir = APPDATA_DIRECTORY

        if FileTest.writable?(dir)
            if creator
                dir = File.join(dir, creator.gsub(RESTRICTED_CHARACTERS, ''))
                Dir.mkdir(dir) if !File.directory?(dir)
            end
            if name
                dir = File.join(dir, name.gsub(RESTRICTED_CHARACTERS, ''))
                Dir.mkdir(dir) if !File.directory?(dir)
            end
            return dir
        end

        if respond_to(;extension_path) && extension_path
            dir = File.dirname(extension_path)
            if File.directory?(dir) && FileTest.writable?(dir)
                # Make a sub directory to avoid clashes
                dir = File.join(dir, '__appdata__')
                Dir.mkdir(dir) unless File.directory?(dir)
                return dir
            end
        end

        # Alternatively, return a default directory, such as
        #   SU/plugin_data/<creator>/<name>/
        raise RuntimeError.new('Not data directory could be located')
    end

end

Then usage would be something like:


module AuthorModule
    module PluginModule
        EXTENSION = SketchupExtension.new('...', '...')
        # ....
        some_path = File.join(EXTENSION.data_directory, 'settings.txt')
    end
end

TIG

BUT...
No one has still answered my question...
What is wrong with using the container folder for the ENV for the OS's user's 'Temp' folder?
That is easily found for either OS and is writable, like the Temp folder itself...
Once you have the path to the user's 'Temp' folder we can then make an app-specific subfolder to hold our temporary files...
AND from File.dirname(Temp) we can get the container-location into which we can make an app-specific subfolder to hold our 'permanent' files...
I do it without problems on all OS's - the rest of you seem to obsessed with finding 'the exact right folder that we must use', when in truth there are several valid possibilities, and my earlier posts cover much of this too... KISS ?

driven

@tig said:

BUT...
No one has still answered my question... :?

I thought Steve had answered when he gave his brief lesson on why temporary directories exist?

Your 'trick' of hiding your 'cookie' from my computer maintenance routines works a treat for now, but may not after the next release of Safari [in beta] or Mavericks [dev release available].

The writings on the wall, Apple doesn't like Plugins or Cookies and if you recall, they weren't to fond of Flash.

Apples allows and even encourages 'Extensions', but they are meant to fully contained within their Parents 'approved' structures that include the ENV[TMPDIR] and or /tmp depending on what your doing. There are also approved external paths for any launch daemon's, browser cookies or cache's that the Parent app and or it's extensions may require. All these additional paths are meant to be in an info.plist so they can be easily removed at any point in time.

Is there a reason to think your cookie [for example] won't work from File.join(File.dirname(__FILE__), "/Data/.Cookie_Jar/.My_cookie.dat")?

I know the path works, is the problem checking it's the contents from your own Plugins sub-directory?

From a general user perspective that is hidden >> inside a hidden folder >> inside your NameSpaced Plugins Folder >> inside the Parent apps 'Plugins' >> inside the normally hidden ENV[HOME]/Library/....

Is there a need to look for anything if your plugin KNOWs where it is.

When your User decides to remove it, the Parent.app or better still, your Plugin only needs to trash the one folder and there are no lurking remains hidden, inappropriately scattered around the system/usr folders.

john

slbaumgartner

@tig said:

BUT...
No one has still answered my question...
What is wrong with using the container folder for the ENV for the OS's user's 'Temp' folder?

One last try and then I will shut up. Your suggestion is what we used to call a "bad hack". It is a hack because it uses an unofficial trick to work around an issue. It is bad because it is only works today due to oversights and mistakes on the part of the OS developers. The fact that the parent directories of temp locations are writable is an oversight, not by intent. They simply never considered that programmers would do what you suggest, so they didn't bother to lock the gate or sweep the floor. They could decide to secure their house at any time, at which point this technique will fail. Do you really want your plugin users to plague you with support calls when Windows 9 or Mac OS X 10.9 changes the rules? Why not look for an OS-endorsed and stable technique rather than take this risk?

Steve

TIG

The parent-folder of Windows 'Temp' folder - 'Local' - IS clearly intended to be written to...
Many applications already make their own subfolders in it to store data more permanently that the Temp folder itself would allow...

If the MAC-guys don't want you writing to the user's 'T' folder's parent-folder, then they have never said so, and to make it non-writable would be straightforward for them...

I don't consider my Windows suggestion a 'hack' at all... the MAC side is less documented, but to date has always worked too...

@driven
The Plugins directory is NOT always writable by the user - this is always the possibility on a PC and before v2013 so it was on a MAC too.
So we can't confidently write any of our folder/files there - in fact we are now full circle because Fredo started this thread trying to find where to save data outside of this potentially limited tree...
Your ENV[HOME] is MAC only, BUT I agree it could be used for that OS by reconstructing the other tree elements from /Library/ - but then who can be sure they'll also remain 'constant' too ?
The equivalent for the PC is actually ENV["HOMEPATH"], but then returns say ' C:/Users/TIG', from which we'd need to construct the upwards tree towards /AppData/Local/XXX etc... This is more easily got working downwards from TEMP or TMP and makes no assumptions about paths ! It does expect Local and Temp to be writable - which is their purpose...

There are many places we cab write folders and files, knowing that the users will assuredly have permission - like their documents folder, their desktop, their 'temp' folders etc...
With the likes of Fredo's folder/files he wants it somewhere discreet and unnoticed...
The 'cookie.dat' file you refer to was originally written to an app specific subfolder inside 'Temp' or 'T' but the potential for it getting purged led me to move it into the container 'parent-folder' - on a PC that is expected to be written to...