Unicode, UTF8 and Ruby

TIG

I came upon this somewhere... Don't know if it has any ideas that help ?

fredo6

Bonjour Fredo,
Le problème ne vient pas d'un fichier. J'ai des noms de matériaux à récupérer dans un modèle SketchUp, pour les re-exporter vers une feuille Excel. Quand tu récupère le nom d'un matériau pour une face f, f.material renvoie par exemple une chaîne s "béton". Quand tu écris cette chaîne dans le fichier Excel, par exemple fichier.puts(s) tu n'obtiens pas "béton", mais "bÃton", parce que les caractères accentués sont codés sur 2 octets au lieu d'un. Et Ruby n'a pas de méthode pour convertir de l'UTF8 en Unicode.

Then, with the explanation from Todd, I understand why I had problem with the dialog boxes, as Windows does support UTF8.

Didier Bur

@unknownuser said:

Don't know if it has any ideas that help

TIG, it seems the "register" method is missing. Apparently not a standard method...

TIG

But couldn't we (you!) use the pack / unpack tricks to convert between the two encoding ?

thomthom

@unknownuser said:

UTF8 doesn't work with the SU Ruby API. I figured out this sad bit of news when I wrote the 3DTextTool.

UTF8 works in Ruby just fine.

Google knows. They've known for the past several maintenance updates.

Todd

That was my first problem when I first tried to write ruby plugins; writing in UTF-8. From doing websites I've grown into the custom of using UTF-8 to account for most languages. I figured that I was doing something wrong and meant to go back and have another look at some point. So, essentially UTF-8 is no-go? And this is due to the SU API - not Ruby?

Didier Bur

Good advice, thanks TIG

TIG

Yes. You can't use FileTest.exist?(Sketchup.active_model.path) if the file has unicode. The 'path' SUp reports looks OK with say ascii_chr=233 for 'é', however the FileTest sees the 'é' as a unicode and so returns false - although they both 'look' the same, the character encoding is different.
My clunky fix only works on the top-most file (or folder) containing the unicode parts, as the Dir.entities(dir) falls over if there are accents earlier in the path...
It can't be beyond the wit of man to take 'Sketchup.active_model.path' and encode it as unicode in a way that would match the Ruby built-ins like FileTest.exist?(path) or Dir.entities(dir)... however it is beyond the wit of me...

thomthom

I think that I couldn't even get UTF-8 scripts to run... I'll have a look at Ruby + SU + UTF. Wonder if Ruby has some nice encoding methods.
Seeing how there's many scripts that uses localisation it's be very nice to have UTF-8.
.SKP has a weird combination of UTF+8 and regular ACSII. Seems that it wasn't originally UTF-8 and it was later added. Maybe we're running into problems due to this.

TIG

233.chr ### a plain ascii é
é
233.chr+233.chr ### 2 number plain ascii é make éé
éé
195.chr ### a plain ascii capital A with an umlaut
Ã
169.chr ### a plain ascii the (c)opyright symbol

195.chr+169.chr ### BUT these 2 number ascii codes added together = one unicode é that looks like an ascii é !!!
é

??? go figure ???

thomthom

UTF only uses two bytes for some of the characters. For most of the latin characters it uses 1byte equal to normal ASCII.

TIG

Didieret al...

After more than a year and a bit...

typical usage: file_found?(Sketchup.active_model.path)

returns trueif the file found,

even with accented unicode characters in name/path,

e.g. qualisé.skp

EDIT: see here for latest file... http://forums.sketchucation.com/viewtopic.php?p=169225#p169225

TIG

file_found?(path) that fixes ascii in SUp Ruby path and unicode in returned filepath returning false negatives with 'FileTest.exist?(path)' - even with accented characters - is updated and moved here... http://forums.sketchucation.com/viewtopic.php?p=169225#p169225