JSON in Ruby

Dan Rathbun · 3 Dec 2012, 14:15

Disable ALL plugins and libraries, (Sketchup::plugins_disabled=true , close SketchUp and restart,) then try again ...

If it works, then you've installed a plugin that changes the String base class. A very big no-no, in a shared Ruby environment, such as SketchUp embedded Ruby (or DoubleCAD XT embedded Ruby, etc.)

In a normal "system" Ruby script, changing a base class doesn't matter because (99% of the time,) a script runs in it's own temporary Ruby process, finishes, and the Ruby process exits.

So.. the most likely culprit would be a library that is written to run temporarily in "system" Ruby, such as Yajl, or the standard JSON library, that changes the String class, or the global methods in mixin module Kernel (such as puts,) or even stream classes, like IO.

Encoding. It is set to UTF-8 by SketchUp just after it loads the interpreter. It should stay that way. Changing it via $KCODE is not a good idea as that (editable) feature has been removed in newer Ruby versions. (You'd just set your code up to fail in a newer SketchUp version if it distro's with Ruby v1.9.x)

thomthom · 3 Dec 2012, 14:48

@myhand said:

This works for me today on my work PC. So must be something specific with my Sketchup installation/version at home. Both are Version 8 out of the box installations on Windows. The one that corrupts the "'" char is the Pro addition, while the one at work is the Standard edition.

Do you have the SketchUp Developer Tools installed? (https://github.com/thomthom/sketchup-developer-tools ) Might be that it's interfering.

Try with just your plugin installed.

Myhand · 3 Dec 2012, 15:45

Thanks guys, I will give this a go tonight. Yes I do have the SketchUp Developer Tools installed.

Dan Rathbun · 3 Dec 2012, 17:24

@myhand said:

... is ComponentDefinition name field. Something like "Bath 6'x34"x54"".

OK.. yea that reminds me. We have similar problems with SketchUp's formatted Length strings when saving things into the registry with Sketchup::write_default() and Sketchup::read_default() methods.

There are some topics on that and using gsub() and tr() before writing values.
The situation is similar because we also get these values with embedded single quotes from the application.

Myhand · 3 Dec 2012, 21:05

@thomthom said:

@myhand said:

This works for me today on my work PC. So must be something specific with my Sketchup installation/version at home. Both are Version 8 out of the box installations on Windows. The one that corrupts the "'" char is the Pro addition, while the one at work is the Standard edition.

Do you have the SketchUp Developer Tools installed? (https://github.com/thomthom/sketchup-developer-tools ) Might be that it's interfering.

Try with just your plugin installed.

Yup it was the SketchUp Developer Tools. Disabling that fixed the problem thanks.

Myhand · 4 Dec 2012, 00:02

OK, with the single quote now working, I have run into the next (and hopefully the last) problem. It appears as if the

WebDialog.execute_script

method strips all \ characters from its arguments as it passes it into JavaScript. I think it is because it does a eval() call on the JS.

e.g.

js_command = "material_maintenance.Sketchup.callUI(\"a string >\\< with a \\\" in the middle\")"; puts "js_command = #{js_command}";

produces this in the Ruby console:


js_command = material_maintenance.Sketchup.callUI("a string >\< with a \" in the middle")

and this when you print the parameter to the callUI function as it hits JS.


a string >< with a " in the middle

replacing js_command with

js_command = "material_maintenance.Sketchup.callUI(\"a string >\\u005c\\u0022< with a \\u005c\\u0022 in the middle\")"

works and produces the desired output in JS


a string >\"< with a \" in the middle

but this feels a bit clunky. Any other ideas?

Aerilius · 4 Dec 2012, 00:49

You need to escape \ characters in the double-quoted javascript code that you want to bring into the webdialog (this first escaping is what makes a valid Ruby string).
%(#000000)["alert('C:\users')"] # bad Ruby string
%(#000000)["alert('C:\\users')"] # good Ruby string
In the webdialog, the string arrives written in a script element. As always, the same escaping rules apply again:
%(#000000)[alert('C:\users')] // bad JavaScript code
%(#000000)[alert('C:\\users')] // good JavaScript code

=> So finally this means we need double escaping on the Ruby side!
%(#000000)["alert('C:\\\\users')"] # Ruby string

P.S: SketchUp does not offer error handling for syntax errors in the Ruby-to-JavaScript string. Assuming my js contains user generated data with unknown characters, or is broken (truncated), we will definitely get a popup about "Syntax Error" that we cannot suppress and handle in other ways. In order to get control of such errors, we could send the code as a JavaScript string and eval() it:
%(#000000)[webdialog.execute_script("try{eval(\"}bad_$yn7aX{\")}catch(e){}")]

All of this is an old story and to many people have suffered, discovered, solved this and re-invented the wheel. For an all-in-one solution that covers this and many other WebDialog issues (for example you will discovered that when sending \\\ from JS to Ruby, it randomly drops some ), please take a look and make use of the WebDialogX project.

thomthom · 4 Dec 2012, 08:43

@myhand said:

method strips all \ characters from its arguments as it passes it into JavaScript. I think it is because it does a eval() call on the JS.

Almost - it injects a SCRIPT element in the webdialog - does pretty much the same thing.
So when you pass data to the WebDialog and something is amiss - check the string you're sending to the WebDialog - is it valid? What throws people off is that you have the quoting needed to create the Ruby string with the JS - and within the JS string you need to follow JS's own quoting and escaping rules.

(Because execute_script adds a SCRIPT element for every call I prefer to remove the SCRIPT element on the JS side afterwards - as I don't like the idea of the DOM tree flooded with SCRIPT elements. I've yet to experience problems with it - but there's something about it that makes my OCD-nerve tingle.)

Myhand · 4 Dec 2012, 12:00

@aerilius said:

=> So finally this means we need double escaping on the Ruby side!
%(#000000)["alert('C:\\\\users')"] # Ruby string

I am sure I tried this last night, but will do some further tests again tonight.

@aerilius said:

In order to get control of such errors, we could send the code as a JavaScript string and eval() it:
%(#000000)[webdialog.execute_script("try{eval(\"}bad_$yn7aX{\")}catch(e){}")]
[/size]

Nice idea, I might use this going forward.

@aerilius said:

please take a look and make use of the WebDialogX project.

I cannot access the content. I get "You do not have access to the wiki." message, even after I have registered.

Dan Rathbun · 4 Dec 2012, 15:06

@myhand said:

@aerilius said:

please take a look and make use of the WebDialogX project.

I cannot access the content. I get "You do not have access to the wiki." message, even after I have registered.

Sorry, it is a 5-slot private repository at this time, and we really need very experienced Rubyists for the slots.

And you are kinda re-inventing the wheel. Js has %(#8000BF)[escape()] and %(#8000BF)[unescape()], and Ruby has them in the URI library.

See topic: http://sketchucation.com/forums/viewtopic.php?f=180&t=49183#p442102

In fact I would vote for Ruby's URI::Escape module to be mixed into the SketchUp API UI::WebDialog class.

Myhand · 4 Dec 2012, 16:27

@dan rathbun said:

Sorry, it is a 5-slot private repository at this time, and we really need very experienced Rubyists for the slots.

No problem Dan, I only looked there as Aerilius recommended I do so.

@dan rathbun said:

And you are kinda re-inventing the wheel. Js has %(#8000BF)[escape()] and %(#8000BF)[unescape()], and Ruby has them in the URI library.

See topic: http://sketchucation.com/forums/viewtopic.php?f=180&t=49183#p442102

In fact I would vote for Ruby's URI::Escape module to be mixed into the SketchUp API UI::WebDialog class.

Thank you, I will try the URI library (I already use escape() in my latest JS code). I do not really want to reinvent the wheel, hence me asking the question here.

Myhand · 5 Dec 2012, 10:00

@aerilius said:

You need to escape \ characters in the double-quoted javascript code that you want to bring into the webdialog (this first escaping is what makes a valid Ruby string).
%(#000000)["alert('C:\users')"] # bad Ruby string
%(#000000)["alert('C:\\users')"] # good Ruby string

In the webdialog, the string arrives written in a script element. As always, the same escaping rules apply again:
%(#000000)[alert('C:\users')] // bad JavaScript code
%(#000000)[alert('C:\\users')] // good JavaScript code

=> So finally this means we need double escaping on the Ruby side!
%(#000000)["alert('C:\\\\users')"] # Ruby string

Thanks for a very clear explanation. I tried this last night and it worked. Need 7 '''s for a "!

thomthom · 5 Dec 2012, 10:05

@myhand said:

Need 7 '''s for a "!

What?

Myhand · 5 Dec 2012, 10:38

@dan rathbun said:

And you are kinda re-inventing the wheel. Js has %(#8000BF)[escape()] and %(#8000BF)[unescape()], and Ruby has them in the URI library.

See topic: http://sketchucation.com/forums/viewtopic.php?f=180&t=49183#p442102

Thanks for this Dan! It works well, and I do not need to remember the number of slashes.

I see that escape() and unescape() are deprecated though and that you are recommended to use

decodeURI()

decodeURIComponent()

Which do you recommend or should I continue with unescape()?

Myhand · 5 Dec 2012, 10:49

@thomthom said:

@myhand said:

Need 7 '''s for a "!

What?

Sorry, meant you need

"a JSON escaped double quote need to be specified like this \\\\\\\" in Ruby for a WebDialog call"

to produce

"a JSON escaped double quote need to be specified like this \" in Ruby for a WebDialog call"

as the parameter of the JS function called.

Dan Rathbun · 5 Dec 2012, 11:40

Hmmm..

given a Js function named say()

.. and a Ruby string:
jarg = %q('a JSON escaped double quote need to be specified like this %q(\") in Ruby for a WebDialog call')

I usually do something like this:

dlg.execute_script("say(#{jarg});")

or I will leave the single quotes out of the jarg string, and put them in at the call ...

jarg = %q(a JSON escaped double quote need to be specified like this %q(\") in Ruby for a WebDialog call.)
dlg.execute_script( "say('" << jarg << "');" )

Dan Rathbun · 5 Dec 2012, 12:53

[quote="Myhand":31wco2xd]I see that %(#8000BF)[escape()] and %(#8000BF)[unescape()] are deprecated (in Javascript,) though, and that you are recommending to use [them]
***%(#BF4000)[
@unknownuser said:

]***

@unknownuser said:

](http://msdn.microsoft.com/en-us/library/dz4x90hk(v) ":31wco2xd]The unescape function should not be used to decode Uniform Resource Identifiers (URI). Use decodeURI and decodeURIComponent functions instead.
%(#8000BF)[decodeURI()]
%(#8000BF)[decodeURIComponent()]

Which do you recommend or should I continue with %(#8000BF)[unescape()]?

The old functions are ASCII, the new ones are Unicode.

Taking a look at the most recent released ECMA-262 (but not the latest proposed revision,) the old functions are no longer listed.

see: ECMA-262, 5.1, Global Object: 15.1.3 URI Handling Function Properties
ECMAScript Language Specification
Standard ECMA-262
5.1 Edition / June 2011

Link to the downloadable PDF of the specification .

But you need to handle the situation where an older browser does not have these functions so, write a wrapper in JS:

var unesc = function(uri) {
    var test = false;
    if (typeof(decodeURI) == "function") test = true;
    return test ? decodeURI(uri) ; unescape(uri);
}