Webdialog to ruby encoding issue on OSX

Dan Rathbun

@jiminy-billy-bob said:

I think I've read something about this a while ago, but can't find it anymore. If anyone has a link...?

Probably this:

WebDialog encoding bug found!

driven

I think it is the thread dan points to...

converting to unicode literal based on the string before retrieving/sending was the only thing that worked for me...

the js from the test rb


      /* Creates a uppercase hex number with at least length digits from a given number */
      function fixedHex(number, length){
          var str = number.toString(16).toUpperCase();
          while(str.length < length)
              str = "0" + str;
          return str;
      }

      /* Creates a unicode literal based on the string. nts; UTF-8 is an encoding - Unicode is a character set*/
      function unicodeLiteral(str){
          var i;
          var result = "";
          for( i = 0; i < str.length; ++i){
              /* You should probably replace this by an isASCII test */
              if(str.charCodeAt(i) > 126 || str.charCodeAt(i) < 32)
                  result += "\\\\" + "u" + fixedHex(str.charCodeAt(i),4);
              else
                  result += str[i];
          }
    return result;
}

john

jiminy-billy-bob

This is the one, thanks guys!

jiminy-billy-bob

John, your solution works well when sending the string back to the webdialog. But in my case I'm trying to read the string inside ruby (actually change layers names).
The problem is that ruby reads the unicode as is, without converting to actual characters. I end up with layers called something like "\u00E9" in SU's layer window (But it displays fine in my webdialog)

I can't find a solution on google. Do you guys have any idea what I should do?

tt_su

@jiminy-billy-bob said:

The problem is that ruby reads the unicode as is, without converting to actual characters.

Is it execute_script being the issue? (Sorry, loooong thread - got confused.)

driven

@jiminy-billy-bob said:

The problem is that ruby reads the unicode as is, without converting to actual characters. I end up with layers called something like "\u00E9" in SU's layer window

are you single quoting it?

> "\u00E9"
é
> '\u00E9'
\u00E9
> %q(\u00E9)
\u00E9
> %Q(\u00E9)
é

or maybe use .inspect?

"\u00E9".inspect
"é"

else
post a snippet, and I'll run some checks...

I did write a ruby encode/decode, but didn't need it for what I was doing...

I'll see if I kept it

john

tt_su

Is your HTML in UTF-8 and tagged with a UTF-8 meta tag so the HTML engine knows to use UTF8?
I tried that ç character in SKUI and it renders fine:

I'm not doing anything special, other than ensuring the my RB files are UTF-8 encoded, that my HTML files are UTF-8 encoded (with UTF-8 characterset META tag.)

tt_su

Ditto on OSX:

I think we need a full example of the error you get - packaged up as RBZ.

tt_su

Btw, what is the source for this character?

driven

@TT

the third last post from the other thread has a rbz that shows the issue on a mac...
http://sketchucation.com/forums/viewtopic.php?f=180%26amp;t=57074%26amp;start=30#p518959

returns using 'get_element_value' have bad encoding regardless of html declarations, script encodings, etc...

something is happening internally in SU that screws things up...

try the rbz it's harmless...

john

Dan Rathbun

Did someone file a bug report ?

driven

@dan rathbun said:

Did someone file a bug report ?

I was going to but I got distracted by work and it's still on my endless todo list...

john

tt_su

@driven said:

the third last post from the other thread has a rbz that shows the issue on a mac...
http://sketchucation.com/forums/viewtopic.php?f=180%26amp;t=57074%26amp;start=30#p518959

So this only happen on OSX, not on Windows?

driven

@tt_su said:

So this only happen on OSX, not on Windows?

as far as I'm aware it's a mac thing...

I think it's because the internal bash env locale defaults to "C" if not implicedly set

> %x(locale)
LANG=
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

whereas same call in 'Terminal.app'

LANG="en_US"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL="en_US.UTF-8"

because it is set...
john

jiminy-billy-bob

@driven said:

post a snippet, and I'll run some checks...

Going back to your test here

If you add

Sketchup.active_model.layers.add param_fix

in @dlg2.add_action_callback("trans_L8_fix")

The layer name is displayed "élan 勢い Schwung импульс" in Layers Panel, as it is a webdialog.
But in Sketchup's layer window, it's displayed "\u00E9lan \u52E2\u3044 Schwung \u0438\u043C\u043F\u0443\u043B\u044C\u0441", just like what's printed in the ruby console.

Any thought on that?

TT > Yes, everything is set to UTF-8. Encoding, meta-tags

slbaumgartner

@driven said:

@tt_su said:

So this only happen on OSX, not on Windows?

as far as I'm aware it's a mac thing...

I think it's because the internal bash env locale defaults to "C" if not implicedly set

> %x(locale)
> LANG=
> LC_COLLATE="C"
> LC_CTYPE="C"
> LC_MESSAGES="C"
> LC_MONETARY="C"
> LC_NUMERIC="C"
> LC_TIME="C"
> LC_ALL=

whereas same call in 'Terminal.app'

LANG="en_US"
> LC_COLLATE="en_US.UTF-8"
> LC_CTYPE="en_US.UTF-8"
> LC_MESSAGES="en_US.UTF-8"
> LC_MONETARY="en_US.UTF-8"
> LC_NUMERIC="en_US.UTF-8"
> LC_TIME="en_US.UTF-8"
> LC_ALL="en_US.UTF-8"

because it is set...
john

I think this is because SU does not set ENV["LANG"] when launching the subshell.

Steve

driven

@ Steve
do you think the two are unrelated?

@jiminy-billy-bob said:

Any thought on that?

if I eval it double quoted, I get élan 勢い Schwung импульс

    puts (eval('"' + param_fix + '"'))

works in UI.messagebox that way as well.

alternatively I wrote [so it is possible] a simple decode method by reversing the JS function back in ruby, it worked but i can't find it...

john

jiminy-billy-bob

eval works great, thanks!

slbaumgartner

@driven said:

@ Steve
do you think the two are unrelated?
john

It's possible they have the same cause, but I am not sure how to prove that.

The man page for the locale command says:

 LANG         Used as a substitute for any unset LC_* variable.  If LANG is unset, it will
              act as if set to "C".  If any of LANG or LC_* are set to invalid values, locale
              acts as if they are all unset.

So the behavior you get is consistent with LANG not being set by SU when it launches a subshell. Also, if you set ENV[LANG] to a specific locale such as "en_US.UTF-8" in the Ruby Console before doing the %x, the other variables all follow.

So far as I can tell, SU doesn't launch a separate process for a WebDialog (at least I don't see one reported in Activity Monitor). I don't know what environment a WebDialog gets...can it be read from javascript?

Steve

driven

@tt_su said:

So this only happen on OSX, not on Windows?

hi TT

did you get a chance to run this on your mac?

and did it confirm what I state, on your setup?

john