WebDialog encoding bug found!
-
@driven said:
@slbaumgartner said:
... Further, it believes this string and the one from get_element_value (that prints as Ãlan) are equal. This makes no sense to me...
I create this confusion... I don't check for equality it's a 'puts', it should be
@dlg2.execute_script("translation3.textContent='#{param_val}' + ' == ' + '#{elm_val}' + ' is ' + '#{param_val == elm_val}'") >
which returns
√©lan == élan is false
sorry I'll get my coat...
john
Nah, I should know enough to check your code . With a real test, equality fails in all cases (as it should!).
-
@slbaumgartner said:
Nah, I should know enough to check your code :oops: . With a real test, equality fails in all cases (as it should!).
it's the tangents that count...
now you can write a proper test...
john
-
The text that displays in your test, says UFT-8 in many places. It is UTF-8.
(The word format is at the end, of Unicode Transformation Format.)(nag, nag)
-
@slbaumgartner said:
But in 2014, the original 5-byte UTF-8 string is somehow transmuted into the 8-byte string \xE2\x88\x9A\xC2\xA9lan. ... It's as if the callback processing code has an incorrect implementation of the transcoding.
YES, agree.
To me it looks like it IS UTF-8, but Ruby thinks it is some other encoding, and doubly transcodes* it into UTF-8 AGAIN.
- P.S. - isn't transmute a math term ?
-
@driven said:
except,
get_element_value
fails pre v2013 andget_element_value
works.? It does? Are you sure?
-
@dan rathbun said:
To me it looks like it IS UTF-8, but Ruby thinks it is some other encoding, and doubly transcodes* it into UTF-8 AGAIN.
That's a similar problem we have with FILE $LOAD_PATH, $LOADED_FEATURES and ENV. UTF-8 byte sequences isn't labeled with the correct encoding.
-
@dan rathbun said:
@slbaumgartner said:
But in 2014, the original 5-byte UTF-8 string is somehow transmuted into the 8-byte string \xE2\x88\x9A\xC2\xA9lan. ... It's as if the callback processing code has an incorrect implementation of the transcoding.
YES, agree.
To me it looks like it IS UTF-8, but Ruby thinks it is some other encoding, and doubly transcodes* it into UTF-8 AGAIN.
- P.S. - isn't transmute a math term ?
I was thinking more of alchemy
-
@tt_su said:
? It does? Are you sure?
It certainly looks like it...
I cleaned up my initial code an per the Nanny and the Professors prodding...
and these are the new images...
and the new script# encoding; UTF-8 def show_problem @lang_hash = {'lid1'=>"élan"} # I'm using a hash because it's what I use in my plugin... @dlg2 = UI;;WebDialog.new("Problem_Main", false,"main_prob", 700, 500, 600, 0, false) html = %Q( <!DOCTYPE html> <html> <head> <title>Problem_Main</title> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"> </head> <body> <p>Tested using Sketchup.version #{Sketchup.version}</p> <p>Tested on #{RUBY_PLATFORM =~ /(darwin)/ ? ((%x(sw_vers).sub(/ProductName;/,'').sub(/ProductVersion;/,'').sub(/BuildVersion;/,'_'))) ; 'windows'}</p> <p>click into the box and hit return or change to use another non ASCII-8BIT string...</p> <input id="#{@lang_hash.keys[0]}" value="#{@lang_hash.values[0]}" type="text" onkeydown="if (event.keyCode == 13) trans_L8();" title="type text and then 'Enter'"> <p>below is the return using 'get_element_value'</p> <h4 id="translation1"><!-- return appears here --></h4> <p>below is the return using 'add_action_callback' </p> <h4 id="translation2"><!-- return appears here --></h4> <p>below is the return of the query 'add_action_callback' == 'get_element_value'</p> <h4 id="translation3"><!-- return appears here --></h4> <script type="text/javascript" charset="UTF-8"> function trans_L8() { window.location = 'skp;trans_L8@'+#{@lang_hash.keys[0]}.id+','+(#{@lang_hash.keys[0]}.value); } </script> </body> </html> ) @dlg2.set_html html RUBY_PLATFORM =~ /(darwin)/ ? @dlg2.show_modal() ; @dlg2.show() @dlg2.add_action_callback("trans_L8") {|dialog, params| param_id = params.split(',')[0].to_s param_val = params.split(',')[1].to_s callback = 'return using add_action_callback ' # + param_id.to_s + ' = ' + param_val.to_s # commented out to avoid the error puts callback p(params) elm_val = (@dlg2.get_element_value(param_id)).to_s element_value = 'return using get_element_value => ' + elm_val puts element_value @dlg2.execute_script("translation1.textContent='#{(elm_val)}'") @dlg2.execute_script("translation2.textContent='#{param_val}'") @dlg2.execute_script("translation3.textContent='#{elm_val == param_val}'") } end show_problem # load("/Users/johns_iMac/Library/Application Support/SketchUp 2014/SketchUp/Plugins/jcb_ViewPortResize/dev/show_encoding_issue.rb") # load("[add your path]/show_encoding_issue.rb")
and a question? Is there code to get the Windows operating system details?
john
-
a CLUE perhaps...
Using js codepoints for my input, 'get_element_value' has switched all the separators...
return using add_action_callback "lid1,/uD83D/uDC7D/u20AC/u00A3/u0061/u0009" return using get_element_value => \uD83D\uDC7D\u20AC\u00A3\u0061\u0009
so I modified my code
@dlg2.add_action_callback("trans_L8") {|dialog, params| param_id = params.split(',')[0].to_s param_val = params.split(',')[1].to_s param_val_sub = params.split(',')[1].gsub('/','\\').to_s p param_val_sub callback = 'return using add_action_callback ' # + param_id.to_s + ' = ' + param_val.to_s # commented out to avoid the error puts callback p(params) elm_val = (@dlg2.get_element_value(param_id)).to_s element_value = 'return using get_element_value => ' + elm_val puts element_value @dlg2.execute_script("translation1.textContent='#{elm_val}'") @dlg2.execute_script("translation2.textContent='#{param_val}'") @dlg2.execute_script("translation3.textContent='#{param_val_sub}'") }
and this is the result
john -
So it seems there are in-fact two issues...
1: return having separators reversed
2: encoding being mis-read
I added the
.gsub('/','\\')
to both types of return, so it works on v8 or v13/v14I added js to convert the string to UNICODE on keydown, before it retrieved by SU...
and the full workaround...
# encoding; UTF-8 def show_problem @lang_hash = {'lid1'=> %Q(élan ümlet)} # I'm using a hash because it's what I use in my plugin... @dlg2 = UI;;WebDialog.new("Problem_Main", false,"main_prob", 700, 500, 600, 0, false) html = %Q( <!DOCTYPE html> <html> <head> <title>Problem_Main</title> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"> </head> <body> <p>Tested using Sketchup.version #{Sketchup.version}</p> <p>Tested on #{RUBY_PLATFORM =~ /(darwin)/ ? ((%x(sw_vers).sub(/ProductName;/,'').sub(/ProductVersion;/,'').sub(/BuildVersion;/,'_'))) ; 'windows'}</p> <p>click into the box and hit return or change to use another non ASCII-8BIT string...</p> <input id="#{@lang_hash.keys[0]}" value="#{@lang_hash.values[0]}" type="text" onkeydown=" if (event.keyCode == 13) this.value=unicodeLiteral(this.value);trans_L8();" title="type text and then 'Enter'"> <p>below is the return using 'get_element_value'</p> <h4 id="translation1"><!-- return appears here --></h4> <p>below is the return using 'add_action_callback' </p> <h4 id="translation2"><!-- return appears here --></h4> <h4 id="translation3"><!-- return appears here --></h4> <script type="text/javascript" charset="UTF-8"> function trans_L8() { window.location = 'skp;trans_L8@'+#{@lang_hash.keys[0]}.id+','+(#{@lang_hash.keys[0]}.value); } /* Creates a uppercase hex number with at least length digits from a given number */ function fixedHex(number, length){ var str = number.toString(16).toUpperCase(); while(str.length < length) str = "0" + str; return str; } /* Creates a unicode literal based on the string */ function unicodeLiteral(str){ var i; var result = ""; for( i = 0; i < str.length; ++i){ /* You should probably replace this by an isASCII test */ if(str.charCodeAt(i) > 126 || str.charCodeAt(i) < 32) result += "\\\\" + "u" + fixedHex(str.charCodeAt(i),4); else result += str[i]; } return result; } </script> </body> </html> ) @dlg2.set_html html RUBY_PLATFORM =~ /(darwin)/ ? @dlg2.show_modal() ; @dlg2.show() @dlg2.add_action_callback("trans_L8") {|dialog, params| param_id = params.split(',')[0].to_s param_val = params.split(',')[1].gsub('/',"\\").to_s callback = 'return using add_action_callback ' # + param_id.to_s + ' = ' + param_val.to_s # commented out to avoid the error puts callback p(params) elm_val = (@dlg2.get_element_value(param_id)).gsub('/',"\\").to_s element_value = 'return using get_element_value => ' + elm_val puts element_value @dlg2.execute_script("translation1.textContent='#{elm_val}'") @dlg2.execute_script("translation2.textContent='#{param_val}'") } end show_problem # load("/Users/johns_iMac/Library/Application Support/SketchUp 2014/SketchUp/Plugins/jcb_ViewPortResize/dev/show_encoding_issue.rb") # load("[add your path]/show_encoding_issue.rb")
BIG question is will this work on other PC's, or would I need a platform conditional?
In review,
I think the http header has the incorrect encoding and as SU is the server it must come from there...
The internal 'separator' variations is still a mystery...john
-
@driven said:
... and a question? Is there code to get the Windows operating system details?
Get the default system Encoding:
Encoding::find("filesystem")
or
Encoding::find("locale")
(On my machine it returns the
#<Encoding:Windows-1252>
object reference.)If you want the Windows version:
%x[ver]
On my machine it returns:
"Microsoft Windows [Version 6.1.7601]"
XP is 5.1
Vista is 6.0
Win7 is 6.1What else do you want to know ?
-
@dan rathbun said:
What else do you want to know ?
does my last script run on your PC?
if swap these in, can you show me a screen shot?
a more complex input...@lang_hash = {'lid1'=> %Q(élan 勢い Schwung импульс)} # I'm using a hash because it's what I use in my plugin...
and the widows versioning...
<p>Tested on #{RUBY_PLATFORM =~ /(darwin)/ ? ((%x(sw_vers).sub(/ProductName;/,'').sub(/ProductVersion;/,'').sub(/BuildVersion;/,'_'))) ; %x[ver]}</p>
also, did you try it without the fix? did you get both returns on v2014?
john
-
I had not actually run the code at all, before.
Here goes:@driven said:
..., did you try it without the fix? did you get both returns on v2014?
On SU2014, without the fix:
load "test/WebDialog_param_encoding_bug.rb" %(#004000)[true return using add_action_callback lid1 = élan return using get_element_value => élan]
- After hitting return the elements are NOT replaced in the WebDialog because
%(#8000BF)[element.textContent=]
does not work on MSIE.
You need a platform code branch and use%(#8000BF)[element.innerText=]
on PC.
On SU2014, WITH the fix, AFTER hitting return:
load "test/WebDialog_param_encoding_fix.rb" %(#004000)[true return using add_action_callback "lid1,\\u00E9lan \\uF8FF \\u00FCmlet" return using get_element_value => \u00E9lan \uF8FF \u00FCmlet]
First setting:
Encoding::default_internal="UTF-8" %(#404000)[UTF-8]
.. has not effect (no difference.) - After hitting return the elements are NOT replaced in the WebDialog because
-
With the two changes on SU2014.
Before hitting ENTER:
.. but when I click in the text control, and hit the END key, the ruby console shows:
%(#004000)[return using add_action_callback "lid1,\xC3\xA9lan \xE5\x8B\xA2\xE3\x81\x84 Schwung \xD0\xB8\xD0\xBC\xD0\xBF\xD1\x83\xD0\xBB\xD1\x8C\xD1\x81" return using get_element_value => élan 勢い Schwung импульс]
After hitting ENTER:
.. and then the Ruby console shows:
%(#004000)[return using add_action_callback "lid1,\\u00E9lan \\u52E2\\u3044 Schwung \\u0438\\u043C\\u043F\\u0443\\u043B\\u044C\\u0441" return using get_element_value => \u00E9lan \u52E2\u3044 Schwung \u0438\u043C\u043F\u0443\u043B\u044C\u0441]
-
thanks dan
@unknownuser said:
You need a platform code branch and use element.innerText= on PC.
innerText works on mac, so I'll just change that...
can you run with the tweak...
and see if they show in the dialog...the extra // in your console return confuses me. I had to add more to escape ruby escaping javascript...
john
-
OK, that latest tweek, after hitting enter:
And the console, after hitting enter:
%(#004000)[load "test/show_encoding_issue.rb" true return using add_action_callback "lid1,\\u00E9lan \\u52E2\\u3044 Schwung \\u0438\\u043C\\u043F\\u0443\\u043B\\u044C\\u0441" return using get_element_value => \u00E9lan \u52E2\u3044 Schwung \u0438\u043C\u043F\u0443\u043B\u044C\u0441]
BTW.. if the WebDialog has focus after hitting ENTER, and I need to use the ALT key (for ALT+PrintScreen to copy the active window image to the clipboard, so I can save it from Paint.NET,) there is a whole bunch of repeated console output, like 15 times while I held the ALT key down.
-
yeh it works...
@dan rathbun said:
BTW.. if the WebDialog has focus after hitting ENTER, and I need to use the ALT key (for ALT+PrintScreen to copy the active window image to the clipboard, so I can save it from Paint.NET,) there is a whole bunch of repeated console output, like 15 times while I held the ALT key down.
with this should I just add a screen grab and dump it to desktop?
@dlg2.write_image(File.join((File.expand_path("~")), "Desktop", "dlg_test.png"))
what's the PC equiv...
EDIT: for completeness I'll add the unmodified pair back into the mix after sleep...
thanks again dan
john -
@driven said:
... should I just add a screen grab and dump it to desktop?
NO.
Replace the
%(#8000BF)[onkeydown=]
with:
%(#8000BF)[onkeyup=" if (event.keyCode == 13) { this.value=unicodeLiteral(this.value); trans_L8(); }"]
Notice that I enclose BOTH the following statements within a "curly block", so that the test only fires when ENTER key is released (after being pressed.)
The condition will not even be tested while any key is down, including any modifier key. -
FYI: I am one of those who hates programs putting stuff on MY Desktop.
-
@dan rathbun said:
FYI: I am one of those who hates programs putting stuff on MY Desktop.
so do I so I added it as a button...
NEW VERSION...
I split the input method to make it function the same in all versions of SU...
I can't get the raw code too display in the dialog, so here it is for v2014, the others are similarly wrong...
EDIT: made a small adjustment to get 'fixed' to display correctly [puts v p]true return using add_action_callback [id,value] "\xE2\x88\x9A\xC2\xA9lan \xC3\x82\xC3\xA3\xC2\xA2\xE2\x80\x9E\xC3\x85\xC3\x91 Schwung \xE2\x80\x93\xE2\x88\x8F\xE2\x80\x93\xC2\xBA\xE2\x80\x93\xC3\xB8\xE2\x80\x94\xC3\x89\xE2\x80\x93\xC2\xAA\xE2\x80\x94\xC3\xA5\xE2\x80\x94\xC3\x85" return using get_element_value => élan 勢い Schwung импульс return using add_action_callback [id,value_fixed] \u00E9lan \u52E2\u3044 Schwung \u0438\u043C\u043F\u0443\u043B\u044C\u0441 return using get_element_value fixed => \u00E9lan \u52E2\u3044 Schwung \u0438\u043C\u043F\u0443\u043B\u044C\u0441
Please test the new script...with puts fix
john
Advertisement