Safe place to store user-defined parameters

fredo6 · 24 Jun 2013, 15:45

I think I will default to the old method of DEFPARAM_Dir in the SU Plugins directory

thomthom · 24 Jun 2013, 17:03

Using Win32 API calls I've been able to get short (DOS 8.3 style) path name to appdata. I need to run some tests to make sure it works under XP etc.

fredo6 · 24 Jun 2013, 17:06

Trying to recap

1) On Windows
For the non-ascii username, we have a much more general problem, because Temp directory and very likely any files (skp, materials), are going to be stamped with the username in the file paths. So nothing will work. For instance, I prepare the HTML files of webdialog boxes in the Temp directory. This is also where I put some traces.
--> I will ignore this until Sketchup moves to Ruby 1.9.2 which solves the problem (or thomthom to find a solution!). Too bad for users having a non-ascii name!

For parameters, the usage of LOCALAPPDATA and APPDATA is OK. It is based on users and should correspond to writable areas
--> I will use this method

2) on Mac
For the non-ascii username, apparently there is no issue with non-ascii usernames (to be confirmed), but anyway, the Temp dir is independent, writable and only visible to users. In addition, the ~/ notation allows avoiding having the user name in the path
--> NO issue there

For parameters, there are visibility issues with the parent folder of ENV["TMPDIR"]. So the method cannot be used.
--> I will use driven's method based on a root directory ~/Library/Application Support/ with a hidden folder "LIBFREDO6_DATA_Dir" under which I will create the files and subfolders.

It's interesting to see that we have had a long debate on a basic question which we should expect to be covered by Sketchup itself. I think we should formulate it clearly for the Trimble SU team to address it in SU 2014.

Fredo

thomthom · 24 Jun 2013, 17:26

As for OSX, what about this:

plugin_path = Sketchup.find_support_file('Plugins') path = File.join( plugin_path, '..', '..' ) File.expand_path( path )

Making some assumption here, but tested under SU8 and SU2013 on OSX 10.8.3:

/Library/Application Support/Google SketchUp 8 /Users/thomas/Library/Application Support/SketchUp 2013

thomthom · 24 Jun 2013, 17:34

@fredo6 said:

1) On Windows
For the non-ascii username, we have a much more general problem, because Temp directory and very likely any files (skp, materials), are going to be stamped with the username in the file paths. So nothing will work. For instance, I prepare the HTML files of webdialog boxes in the Temp directory. This is also where I put some traces.
--> I will ignore this until Sketchup moves to Ruby 1.9.2 which solves the problem (or thomthom to find a solution!). Too bad for users having a non-ascii name!

On my Win7 systems I tried, the TEMP/TMP ENV variables returned a short (DOS 8.3 style) path name. These short paths was writable with Ruby 1.8 as the system resolved their real paths. (I assume because the short paths where given to the ANSI version of the system file functions which in turn handled it.)

@fredo6 said:

For parameters, the usage of LOCALAPPDATA and APPDATA is OK. It is based on users and should correspond to writable areas
--> I will use this method

There also include username in the path and may be non-writable. In fact - you are more likely to get issues here because the ENV variable doesn't return a short path - but a path string with replaced characters.

@fredo6 said:

2) on Mac
For the non-ascii username, apparently there is no issue with non-ascii usernames (to be confirmed), but anyway, the Temp dir is independent, writable and only visible to users. In addition, the ~/ notation allows avoiding having the user name in the path
--> NO issue there

Unicode filenames works under OSX with Ruby 1.8 because the file system functions calls handles UTF-8 strings.

@fredo6 said:

For parameters, there are visibility issues with the parent folder of ENV["TMPDIR"]. So the method cannot be used.
--> I will use driven's method based on a root directory ~/Library/Application Support/ with a hidden folder "LIBFREDO6_DATA_Dir" under which I will create the files and subfolders.

What about under '~\Library\Application Support\SketchUp 2013' ? That way the data is isolated from version to version.

fredo6 · 24 Jun 2013, 17:50

On my Windows 7,
ENV["TMP"] --> C:\Users\Fred\AppData\Local\Temp
ENV["LOCALAPPDATA"] --> C:\Users\Fred\AppData\Local
So I don't see the short Dos 8.3 notation on my PC

Anyway, since the releasing of my scripts I have been using ENV["TMP"] and I never came across a user having problem with non-ascii username. So maybe very few users do set up their PC with a non-ascii username

On Mac,

It seems SU 13 plugins is installed under ~/Library/Application Support/SketchUP 2013/Sketchup/Plugins.

So I still have a problem to understand

why we could create a directory from Ruby under ~/Library/Application Support
and why it fails when you create it in ~/Library/Application Support/SketchUP 2013/Sketchup/Plugins (i.e. the failure reported on DEFPARAM_Dir creation)

Fredo

driven · 24 Jun 2013, 17:55

@thomthom said:

As for OSX, what about this:

plugin_path = Sketchup.find_support_file('Plugins') path = File.join( plugin_path, '..', '..' ) File.expand_path( path )

The problem is when Sketchup.find_support_file('Plugins') returns the non-user Library

the only way to be sure you have a User writable Plugins Folder is

userPlugins = (File.dirname(File.expand_path(Sketchup.find_support_file("OldColors")))) <<  ("/Plugins")
Dir.mkdir(userPlugins ) unless File.exist?(userPlugins )

This empty Folder OldColors is in every version since v5 at least, it's not dynamically created like Shortcuts.plist nor movable like AutoSave

SU on mac has always been able to have 2 'Plugins' paths, it was mix and match that messed things up...

john

thomthom · 24 Jun 2013, 18:25

@fredo6 said:

On my Windows 7,
ENV["TMP"] --> C:\Users\Fred\AppData\Local\Temp
ENV["LOCALAPPDATA"] --> C:\Users\Fred\AppData\Local
So I don't see the short Dos 8.3 notation on my PC

Because your username "Fred" is compatible with the 8.3 annotation. If however the username is "tæst" the the returned value would be:

ENV["TMP"] --> C:\Users\TST~1\AppData\Local\Temp ENV["LOCALAPPDATA"] --> C:\Users\T'st\AppData\Local

As demonstrated by this example: http://sketchucation.com/forums/viewtopic.php?f=180&t=52730&start=60#p482159

So ENV["LOCALAPPDATA"] is not safe and will fail.

@fredo6 said:

Anyway, since the releasing of my scripts I have been using ENV["TMP"] and I never came across a user having problem with non-ascii username. So maybe very few users do set up their PC with a non-ascii username

That's because ENV["TMP"] returns the path in the short 8.3 format.

I've managed to make Win32 API system calls to get the appdata folder in this format. But I don't see any other way to get this path. For me it's not a big deal because I already ship D. Berger's Win32::API class wrapped under the TT:: namespace in TT_Lib2. But it's not an ideal solution for everyone as it requires bundling a third party library.

fredo6 · 24 Jun 2013, 19:45

Thomthom,

Thanks really for all your efforts.

On Win7, LOCALAPPDATA seems to be simply the parent of the Temp folder.

So maybe the solution is to use for local data File.dirname ENV["TMP"], which hopefully will preserve the 8.3 notation.

It may even work on Windows XP

[EDIT]from Internet googling , it seems that the temp directory on Xp is %SystemDrive%\Documents and Settings{username}\Local Settings\Temp.
On Windows Vista, 7, 8, the directory is %SystemDrive%\Users{username}\AppData\Local\Temp

So the method File.dirname ENV["TEMP"] would work on both Widnows systems, provided this call File.dirname preserves the Dos8.3 notation.

Fredo

thomthom · 24 Jun 2013, 20:11

Under XP:

` ENV['LOCALAPPDATA']
nil

ENV['APPDATA']
C:\Documents and Settings\XPMUser\Application Data`

However - APPDATA != Local AppData - so it's not the correct fallback to make.

ENV variable:

ENV.to_hash { "PROCESSOR_ARCHITECTURE"=>"x86", "HOMEDRIVE"=>"C:", "APPDATA"=>"C:\\Documents and Settings\\XPMUser\\Application Data", "USERPROFILE"=>"C:\\Documents and Settings\\XPMUser", "TMP"=>"C:\\DOCUME~1\\XPMUser\\LOCALS~1\\Temp", "SESSIONNAME"=>"Console", "ProgramFiles"=>"C:\\Program Files", "PROCESSOR_REVISION"=>"1707", "PROCESSOR_LEVEL"=>"6", "CommonProgramFiles"=>"C:\\Program Files\\Common Files", "USERNAME"=>"XPMUser", "PROCESSOR_IDENTIFIER"=>"x86 Family 6 Model 23 Stepping 7, GenuineIntel", "OS"=>"Windows_NT", "FP_NO_HOST_CHECK"=>"NO", "CLIENTNAME"=>"Console", "windir"=>"C:\\WINDOWS", "SystemRoot"=>"C:\\WINDOWS", "PATHEXT"=>".COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH", "ALLUSERSPROFILE"=>"C:\\Documents and Settings\\All Users", "USERDOMAIN"=>"VIRTUALXP-IE8", "NUMBER_OF_PROCESSORS"=>"1", "HOMEPATH"=>"\\Documents and Settings\\XPMUser", "TEMP"=>"C:\\DOCUME~1\\XPMUser\\LOCALS~1\\Temp", "SystemDrive"=>"C:", "Path"=>"C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\Program Files\\TortoiseHg\\", "LOGONSERVER"=>"\\\\VIRTUALXP-IE8", "ComSpec"=>"C:\\WINDOWS\\system32\\cmd.exe", "COMPUTERNAME"=>"VIRTUALXP-IE8" }

When I make Win32 calls to get the path from the system I get:

TT::Win32.get_folder_path_ansi( TT::Win32::CSIDL_LOCAL_APPDATA ) C:\Documents and Settings\XPMUser\Local Settings\Application Data

That is the full path. But not that this is an English XP version and the user is "XPMUser" which all contains ASCII characters. When the OS language changes or username changes to contain non-ASCII characters it will fail.

The 8.3 version of the path - that seem to work better for Ruby 1.8 is:
TT::Win32.get_short_folder_path_ansi( TT::Win32::CSIDL_LOCAL_APPDATA ) C:\DOCUME~1\XPMUser\LOCALS~1\APPLIC~1

But one cannot hard code these either.

And the different folder structure also means one cannot go from TEMP path to the local appdata path.

fredo6 · 24 Jun 2013, 20:33

Forget LOCALAPPDATA and APPDATA.

What I say is that, if ENV["TEMP"] return a DOS8.3 notation, and if File.dirname keeps this Dos8.3 notation, then the root to store the data can be File.dirname(ENV["TEMP"].

It happens that:

On Win7, Vista, Win8, the path calculated with this method will be equivalent to ENV["LOCALAPPDATA"]
on WinXP, I am pretty sure that the DOS8.3 notation C:\\DOCUME~1\\XPMUser\\LOCALS~1 is equivalent to the long notation C:\Documents and Settings\XPMUser\Application Data, which is precisely ENV["APPDATA"]

Fredo

thomthom · 24 Jun 2013, 20:35

@fredo6 said:

on WinXP, I am pretty sure that the DOS8.3 notation C:\\DOCUME~1\\XPMUser\\LOCALS~1 is equivalent to the long notation C:\Documents and Settings\XPMUser\Application Data, which is precisely ENV["APPDATA"]

Problems is localization - I think XP used localized names for the paths. Lemme just find my old XP machine with Norwegian language and verify.

fredo6 · 24 Jun 2013, 20:38

@thomthom said:

Problems is localization - I think XP used localized names for the paths. Lemme just find my old XP machine with Norwegian language and verify.

The script does not really bother as long as ENV["TEMP"] and File.dirname(ENV["TEMP"]) are valid directories with only ASCII characters.

Fredo

thomthom · 24 Jun 2013, 20:45

I'll try to check that.

thomthom · 24 Jun 2013, 20:49

@fredo6 said:

on WinXP, I am pretty sure that the DOS8.3 notation C:\\DOCUME~1\\XPMUser\\LOCALS~1 is equivalent to the long notation C:\Documents and Settings\XPMUser\Application Data, which is precisely ENV["APPDATA"]

APPDATA under XP is:
C:\Documents and Settings\XPMUser\Application Data

C:\DOCUME~1\XPMUser\LOCALS~1
This represent:
C:\Documents and Settings\XPMUser\Local Settings

And Local AppData is
C:\Documents and Settings\XPMUser\Local Settings\Application Data

which is mapped to:
C:\DOCUME~1\XPMUser\LOCALS~1\APPLIC~1

fredo6 · 24 Jun 2013, 21:09

OK. Thanks for the info.

But C:\Documents and Settings\XPMUser\Local Settings can make it as well (possibly I can look for a subfolder Application Data).

I keep in mind that the reason of using a path calculation based on ENV["TEMP"] is to get a valid path in pure ascii.

Actually I am just looking for a location:

which is writable from Ruby
which is stable through time
which is not erased by the system accidentally or intentionally
possibly which can be pointed to via a pure Ascii characters
Otherwise, the name itself does not matter as the user would have no reason to go there.

So, what's left to do is to check whether File.dirname(ENV["TEMP"]) maintains the DOS8.3 pure ascii notation.

Fredo

thomthom · 24 Jun 2013, 23:08

Norwegian XP - User "Tæst":

` ENV.to_hash
{
"PROCESSOR_ARCHITECTURE"=>"x86",
"HOMEDRIVE"=>"C:",
"CLASSPATH"=>".;C:\Programfiler\Java\jre1.6.0_07\lib\ext\QTJava.zip",
"APPDATA"=>"C:\Documents and Settings\T\221st\Programdata",
"USERPROFILE"=>"C:\Documents and Settings\T\221st",
"TMP"=>"C:\DOCUME~1\TST~1\LOKALE~1\Temp",
"SESSIONNAME"=>"Console",
"ProgramFiles"=>"C:\Programfiler",
"PROCESSOR_REVISION"=>"0d08",
"PROCESSOR_LEVEL"=>"6",
"CommonProgramFiles"=>"C:\Programfiler\Fellesfiler",
"USERNAME"=>"T\221st",
"PROCESSOR_IDENTIFIER"=>"x86 Family 6 Model 13 Stepping 8, GenuineIntel",
"OS"=>"Windows_NT",
"FP_NO_HOST_CHECK"=>"NO",
"windir"=>"C:\WINDOWS",
"SystemRoot"=>"C:\WINDOWS",
"PATHEXT"=>".COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH",
"ALLUSERSPROFILE"=>"C:\Documents and Settings\All Users",
"USERDOMAIN"=>"WEEBLE",
"NUMBER_OF_PROCESSORS"=>"1",
"HOMEPATH"=>"\Documents and Settings\T\221st",
"TEMP"=>"C:\DOCUME~1\TST~1\LOKALE~1\Temp",
"SystemDrive"=>"C:",
"QTJAVA"=>"C:\Programfiler\Java\jre1.6.0_07\lib\ext\QTJava.zip",
"Path"=>"C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\PROGRA~1\FELLES~1\SONICS~1\;C:\Programfiler\Fellesfiler\Autodesk Shared\;C:\Programfiler\backburner 2\;C:\Programfiler\Fellesfiler\Roxio Shared\DLLShared;c:\Programfiler\Microsoft SQL Server\90\Tools\binn\;C:\Webdev\MySQL\MySQL Server 4.1\bin;C:\Programfiler\QuickTime\QTSystem\",
"LOGONSERVER"=>"\\WEEBLE",
"ComSpec"=>"C:\WINDOWS\system32\cmd.exe",
"COMPUTERNAME"=>"WEEBLE"
}

TT::System.local_data_path
C:\DOCUME~1\TST~1\LOKALE~1\PROGRA~1

ENV['TEMP']
C:\DOCUME~1\TST~1\LOKALE~1\Temp

File.dirname( ENV['TEMP'] )
C:\DOCUME~1\TST~1\LOKALE~1

TT::Win32.get_folder_path_ansi( TT::Win32::CSIDL_LOCAL_APPDATA )
C:\Documents and Settings\Tæst\Lokale innstillinger\Programdata

TT::Win32.get_short_folder_path_ansi( TT::Win32::CSIDL_LOCAL_APPDATA )
C:\DOCUME~1\TST~1\LOKALE~1\PROGRA~1

path = TT::Win32.get_short_folder_path_ansi( TT::Win32::CSIDL_LOCAL_APPDATA )
C:\DOCUME~1\TST~1\LOKALE~1\PROGRA~1

File.exist? path
true

file = File.join( path, 'test.txt' )
C:\DOCUME~1\TST~1\LOKALE~1\PROGRA~1/test.txt

File.open( file, 'w' ) { |f| f.puts 'Hello World' }
nil

File.exist? file
true

File.size file
13`

PROGRA~1 in the local appdata folder is just a coincidence as the Norwegian work is very similar to English.

C:\Documents and Settings\Tæst\Lokale innstillinger\Programdata

It'd be nice to have this tested on a computer with very different language.