Upgrading plugins to Ruby 2.0 for SketchUp 2014
-
@marksup said:
Does this need to be fixed by SketchUp and/or is there anything I can do to detect/avoid the issue (since I can not control either the selected CSV file content or its encoding)?
Bugsplats should not happen and is something we should fix. Have you submitted the splats? With any information we can use to look it up?
@marksup said:
These UTF-8 errors are insidious, and a definitive guide to their avoidance would be very helpful. What have other Ruby programmers been doing (prior to SketchUp 2014) to adjust to the new Ruby version?, are there no existing resources?
If you search on Ruby in general, without the context of SketchUp, you find a number of articles and forum posts with people migrating from Ruby 1.8 to 1.9/2.0.
The bugsplat here is troublesome, because it should just raise an error. If it had raised an error you would have been able to try again with a different encoding. Now, I haven't read arbitrary files with ruby yet, but I would have thought there would be a methods in the Ruby StdLib to detect encoding of a given string/file?
Back to the bugsplat, can you provide a small snippet that reads a file and a file that will cause it to crash? Smallest possible snippet to reproduce this?
-
If the .CSV file is not in UTF8-without-BOM encoding, but say ANSI, and also since it may include characters like 'm²', these both might cause you issues.
It's easy enough to [re]encode any file appropriately using Notepad++.exe...
However if the 'm²' were typed in say Unicode, then it might then be converted to a strange looking character combo when re-encoded.
In that case you need to retype the character[s] and it should convert automatically to display in UTF8 encoding...
Save and retry.Of course, the .RB file itself must of course be properly encoded as UTF8-without-BOM to be compatible with v2014's Ruby2.0, and remember that any special characters [like 'm²'] which you had in any earlier encoding might have to be corrected if already UTF8 compliant... as outlined above...
-
Notepad++ is great, but customers of plugins can not be expected to use it to investigate and correct the encoding of their data files.
Is the encoding of a scrambled Ruby file significant? - since a scrambled UTF-8 .rb is reported by Notepad++ as ANSI.
-
It is unfortunate that the encoding of a PC's plain Notepad file is in ANSI.
However a txt/csv file made in Excel or through a Ruby script should be encoded as compatible UFT8-without-BOM...If you have made the RBS from an incorrectly encoded RB it will fail in v2014.
If you make the RB correctly encoded than encrypt the RBS from that is should be compatible.
What Notepad++ sees the contents as is somewhat academic as isn't it a binary file ?Files encoded as UFT8-without-BOM should be compatible with v2014 Ruby2.0 AND all earlier versions of SketchUp and their Ruby version...
You could try trapping for the Sketchup.version >= 14 then re-encoding the string got from any data file.
You should still be able to 'read' the contents of an ANSI encoded data file [or any type of data file - even binary is 'readable'], but to then use that string in Ruby2.0 you probably need to force some recoding...if Sketchup.version.to_i >= 14 lines = IO.read(csv_file_path).force_encoding('UTF-8') else lines = IO.read(csv_file_path) end lines.split("\n").each{|line| ### process the line... }I typed this without reference and it is untested, but you get the idea...
-
Making the first statements of the block:
if defined?(Encoding) file_data.encode!("UTF-8") unless file_data.encoding == Encoding;;UTF_8 end
-
You could try to open the file as ASCII-8bit - then attempt to convert the data afterwards. But in general you need to know what encoding a file is in. You'll see editors often provide this option, even Notepad.
After reading the file as ASCII-8bit - which is in effect what Ruby 1.8 did - then you could try to change the encoding to UTF-8 - if that fails it's a good chance the file is ANSI encoded in which you can retry with that. (Though there are many variants of ANSI, US-ANSI is most normal one.)
As for the crashes - as I mentioned, that should not happen. Did you submit those report? This is important for us in order to address the crash.
-
Bugsplat report duly filed.
-
-
I used the original post above as the scenario description, so... Ruby, Importer, UTF-8 and/or m² should find it.
-
hm... this crashes deep into the Ruby interpreter...
Can you provide a small code snippet to reproduce it?
-
I see you included some description in the BugSplat, but I'm afraid it's been mangled formatting. Can you attach sample RB file and sample CSV file? That'll ensure we are reproducing 100% correctly.
-
FWIW. Now in Ruby 2.x, you can specify the arguments that will be used for
IO.newviaIO.open, as the last argument toFile.foreach(), which is inherited from[%(#BF0000)[IO.foreach()]](http://www.ruby-doc.org/core-2.0.0/IO.html#method-c-foreach).Example
File.foreach( csv_filepath, {;mode=>"r", ;encoding=>"ASCII;UTF-8"} ) do |file_data| # statements end -
-
I also wrote:
@dan rathbun said:..., you can specify the arguments that will be used for
IO.newso ...
http://www.ruby-doc.org/core-2.0.0/IO.html#method-c-newIt's an example.
At the console you can get names like this:
Encoding::ASCII.names %(#008040)[>> ["US-ASCII", "ASCII", "ANSI_X3.4-1968", "646"]]Encoding::ASCII_8BIT.names %(#008040)[>> ["ASCII-8BIT", "BINARY"]]
-
I had a weird problem with a txt file using IO.read to get a string...
Notepad++ said was encoded as ANSI, but Ruby2.0 said was was encoded as UTF-8...
I could read and parse the string in Ruby1.8, but in Ruby2.0 it sometime 'threw a wobbler' about wrongly encoded characters...If the string was all normal ASCII type characters then no issue...
BUT if the string contained an accented character it caused the 'wobbler'...If I re-encoded it in Notepad++ to UTF8-without-BOM - then again no issues with the accents in Ruby2.0 or Ruby1.8.
A 'puts' for the strings read for the two file encodings showed differences with \E... etc where the accents were.
In Ruby2.0 I tried to force the encoding to UFT-8, but since it thought it already was in that it failed.
This was my workaround:data = IO.read(@data_file) data = data.force_encoding('ISO-8859-1').encode("UTF-8") if defined?(Encoding) @uid = data.split('&')[-1].to_s.chomp --> Gábor
etc...Now in v2014 it reads and parses OK and in earlier versions it works too...
Forcing the encoding into one that it is never going to be [as in my case anyway] and then back to UTF-8 works...
I don't know why it thought the encoding was NOT in ANSI but...Incidentally - this also uncovered another oddity...
In Ruby1.8 if you use the File.new to make a txt file and write an ASCII string it is reported as a UTF8... encoded file by Notepad++
BUT if the string contains an accented character the file encoding is reported as ANSI.
Since Ruby1.8 can cope with either encoding when parsing strings etc it causes no issues, but if you have such a txt file, without the above double encoding workaround it causes issues with Ruby2.0... -
TIG - THANK YOU!
I was working on the UTF-8 errors issue when your last post displayed. I can confirm that...
File.foreach(csv_filepath) do |file_data|
file_data.force_encoding('ISO-8859-1').encode("UTF-8") if defined?(Encoding)Works in both (SketchUp) Ruby releases exactly as you say.
-
I've reproduced the crash and it happens in the Importer class wrapper that is supposed to return the result code. That's why it works when you call the method directly and not via the Importer dialog.
-
Right, so I got a fix for it on our side. Until then you can work around the crash by catching Ruby errors in Importer.load_file
<span class="syntaxdefault"><br /></span><span class="syntaxkeyword">class </span><span class="syntaxdefault">CSV_ImporterC </span><span class="syntaxkeyword">< </span><span class="syntaxdefault">Sketchup</span><span class="syntaxkeyword">;;</span><span class="syntaxdefault">Importer<br /> def description<br /> </span><span class="syntaxkeyword">return </span><span class="syntaxstring">"CSV Importer (*.csv)"<br /> </span><span class="syntaxdefault">end<br /> def file_extension<br /> </span><span class="syntaxkeyword">return </span><span class="syntaxstring">"csv"<br /> </span><span class="syntaxdefault">end<br /> def id<br /> </span><span class="syntaxkeyword">return </span><span class="syntaxstring">"CSV-Importer"<br /> </span><span class="syntaxdefault">end<br /> def supports_options</span><span class="syntaxkeyword">?<br /> return </span><span class="syntaxdefault">false<br /> end<br /> def load_file</span><span class="syntaxkeyword">(</span><span class="syntaxdefault">file_path</span><span class="syntaxkeyword">, </span><span class="syntaxdefault">status</span><span class="syntaxkeyword">)<br /> return </span><span class="syntaxdefault">1 </span><span class="syntaxkeyword">if ( </span><span class="syntaxdefault">not status</span><span class="syntaxkeyword">)<br /> </span><span class="syntaxdefault">CSV_importerF</span><span class="syntaxkeyword">(</span><span class="syntaxdefault">file_path</span><span class="syntaxkeyword">)<br /> return </span><span class="syntaxdefault">0<br /> rescue StandardError </span><span class="syntaxkeyword">=> </span><span class="syntaxdefault">e<br /> puts e</span><span class="syntaxkeyword">.</span><span class="syntaxdefault">message<br /> puts e</span><span class="syntaxkeyword">.</span><span class="syntaxdefault">backtrace</span><span class="syntaxkeyword">.</span><span class="syntaxdefault">join</span><span class="syntaxkeyword">(</span><span class="syntaxstring">"\n"</span><span class="syntaxkeyword">)<br /> return </span><span class="syntaxdefault">1<br /> end<br />end<br /><br />def CSV_importerF</span><span class="syntaxkeyword">(</span><span class="syntaxdefault">csv_filepath</span><span class="syntaxkeyword">)<br /> </span><span class="syntaxdefault">File</span><span class="syntaxkeyword">.foreach(</span><span class="syntaxdefault">csv_filepath</span><span class="syntaxkeyword">) do |</span><span class="syntaxdefault">file_data</span><span class="syntaxkeyword">|<br /> </span><span class="syntaxdefault">next </span><span class="syntaxkeyword">if ( </span><span class="syntaxdefault">file_data</span><span class="syntaxkeyword">.</span><span class="syntaxdefault">match</span><span class="syntaxkeyword">(/\</span><span class="syntaxdefault">A</span><span class="syntaxcomment">#/) or not file_data.include?(",") )<br /> </span><span class="syntaxdefault">puts </span><span class="syntaxstring">"Floor #{file_data.chomp.split("</span><span class="syntaxkeyword">,</span><span class="syntaxstring">")[1].to_i}"<br /> </span><span class="syntaxdefault">end<br />end<br /><br />Sketchup</span><span class="syntaxkeyword">.</span><span class="syntaxdefault">register_importer</span><span class="syntaxkeyword">(</span><span class="syntaxdefault">CSV_ImporterC</span><span class="syntaxkeyword">.new)<br /> </span><span class="syntaxdefault"></span>As for reading the file correctly, I would first try to read the CSV file in UT-8, then if that fails, fall back to ISO-8859-1 (or maybe try to use whatever codepage is in use if Ruby will let you know this.)
-
Didn't Bugra say the two Ruby loadpaths were encoded in the local code page ?
-
@tt_su said:
I've reproduced the crash and it happens in the Importer class wrapper that is supposed to return the result code. That's why it works when you call the method directly and not via the Importer dialog.
This isn't new for SketchUp 2014 - it's been around for awhile.
Advertisement