JSON in Ruby

Dan Rathbun

Thanx for sharing.

However it has one major drawback. You defined your class at the toplevel. This should be reserved for Ruby Core basclasses.

Why? We all cannot be defining custom classes in the global ObjectSpace, otherwise MY class JSON will clash with YOUR class JSON !

Custom classes should be defined within YOUR "Author" namespace, or sub-namespace, however you wish to organize it.

For example:

module MyHand
  module Lib
    module JSON
      # the code here
    end # module JSON
  end # module Lib
end # module MyHand

Then if anyone (including you,) wishes to use it from within one of their sub-modules, they have several options.

(1) Create a reference that aliases into your lib class:

module Author
  module NiftyPlugin

    # make sure the lib file is loaded
    require("myhand/lib/json.rb")

    # create an alias;
    JSON = MyHand;;Lib;;JSON

    # use it;
    json = JSON.new()

  end # module NiftyPlugin
end #

(2) Make your JSON a MIXIN module instead of a class.
Users then mix it into their nested namespaces:

module Author
  module NiftyPlugin

    # make sure the lib file is loaded
    require("myhand/lib/json.rb")

    # mix in the library module, as a nested class;
    class JSON
      # bring in methods as instance methods;
      include(MyHand;;Lib;;JSON)
      # bring in methods as class methods;
      extend(MyHand;;Lib;;JSON)
    end # class

    # use it calling a class method;
    str = JSON.escape("\tHello World!\n")

  end # module NiftyPlugin
end #

They can do similar by mixing into a nested module.

In fact the way you wrote it, the methods never the ref to the self instance, so basically you wrote class (ie, singleton) methods, so it really IS a library module, but you mis-defined it as a class.

If you change it to a module (within some custom modular namespace,) and then at the top of the module call module_function(), and Ruby will make a copy of each of the methods for you (one instance and one singleton.)

Aerilius

Just for completion, if someone uses Dan's simple example it works well if you know what key/values you have (configurations, options...).
In case you don't know the possible values (ie. user input that might contain colons : or =>), you have to make sure that you don't replace them within strings, only between keys and values.


def from_json(json_string)
  # split at every even number of unescaped quotes; if it's not a string then replace ; and null
  ruby_string = json_string.split(/(\"(?;.*?[^\\])*?\")/).
    collect{|s|
      (s[0..0] != '"')? s.gsub(/\;/, "=>").gsub(/null/, "nil") ; s
    }.
    join()
  result = eval(ruby_string)
  return result
rescue Exception => e
  {}
end


def to_json(obj)
  json_classes = [String, Symbol, Fixnum, Float, Array, Hash, TrueClass, FalseClass, NilClass]
  # remove non-JSON objects
  check_value = nil
  check_array = Proc.new{|o| o.reject!{|k| !check_value.call(k) } }
  check_hash = Proc.new{|o| o.reject!{|k,v| !k.is_a?(String) && !k.is_a?(Symbol) || !check_value.call(v) } }
  check_value = Proc.new{|v|
    if v.is_a?(Array)
      check_array.call(v)
    elsif v.is_a?(Hash)
      check_hash.call(v)
    end
    json_classes.include?(v.class)
  }
  return "null" unless check_value.call(obj)
  # split at every even number of unescaped quotes; if it's not a string then turn Symbols into String and replace => and nil
  json_string = obj.inspect.split(/(\"(?;.*?[^\\])*?\")/).
    collect{|s|
      (s[0..0] != '"')?                        # If we are not inside a string
      s.gsub(/\;(\S+?(?=\=>|\s))/, "\"\\1\""). # Symbols to String
        gsub(/=>/, ";").                       # Arrow to colon
        gsub(/\bnil\b/, "null") ;              # nil to null
      s
    }.join()
  return json_string
end

OricAtmos

Is there a reason to avoid libraries like Yajl, because that's what I'm using. I should note that our plugin isn't distributed in any way and only used internally so far and I just put all necessary Yajl files in a sub folder inside our plugin folder. So that's not too pretty I guess, but it works fine.

Myhand

@oricatmos said:

Is there a reason to avoid libraries like Yajl, because that's what I'm using. I should note that our plugin isn't distributed in any way and only used internally so far and I just put all necessary Yajl files in a sub folder inside our plugin folder. So that's not too pretty I guess, but it works fine.

Thanks! I have not seen Yajl. Will give it a try. I tried

http://flori.github.com/json/doc/index.html

before but could not get it to work within Sketchup hence me writing my own class.

Myhand

@dan rathbun said:

Thanx for sharing.

However it has one major drawback. You defined your class at the toplevel. This should be reserved for Ruby Core basclasses.

In fact the way you wrote it, the methods never the ref to the self instance, so basically you wrote class (ie, singleton) methods, so it really IS a library module, but you mis-defined it as a class.

Thanks Dan, this class is actually part of a module in my code, I just cut and pasted it out to show how it works. Thank you for the tips about libraries though, I have not explored this side of Ruby yet.

You say that my methods are class methods. But I have done some tests and they seem to behave as object methods. i.e. each has access to the object's local state variables. I can also not call them as you would call a static method, i.e. JSON.escape().

Why do you think my methods are static?

Cheers

Myhand

@oricatmos said:

Is there a reason to avoid libraries like Yajl, because that's what I'm using. I should note that our plugin isn't distributed in any way and only used internally so far and I just put all necessary Yajl files in a sub folder inside our plugin folder. So that's not too pretty I guess, but it works fine.

Hi OricAtmos, I tried to get this to work but keep getting errors when trying to include the library:

require 'yajl'

fails with the following error:

load "c:/temp/yajl_test.rb"
Error: #<LoadError: c:/temp/yajl.rb:1:in require': no such file to load -- yajl/yajl> c:/temp/yajl_test.rb:1 c:/temp/yajl.rb:1 c:/temp/yajl_test.rb:1:in require'
c:/temp/yajl_test.rb:1
(eval):1:in `load'
(eval):1

How did you install the library?

OricAtmos

@myhand said:

Hi OricAtmos, I tried to get this to work but keep getting errors when trying to include the library:
[...]

How did you install the library?

I think it might have to do with the library search paths or something. I'll get back to you tomorrow with the details.

Dan Rathbun

@myhand said:

I tried to get this to work but keep getting errors when trying to include the library:
require 'yajl'
If you do not specify some kind of filepath, then the file must be in one of the directories listed in the $LOAD_PATH array.

Ruby's require() first checks to see if the argument resolves to an absolute path, and if so, checks to see if the file exists, and if true, attempts to load it.

Secondly, it checks to see if the argument is a relative path (incl. no path at all,) and if so, require() then iterates the $LOAD_PATH array prepending the base paths in front of your relative path. If it finds match, it loads the file, IF such a path is NOT ALREADY present in the $LOADED_FEATURES (aka $") array.

What does the following LoadError exception message tell you?
Error: #<LoadError: c:/temp/yajl.rb:1:inrequire': no such file to load -- yajl/yajl>`

Answer: That "c:/temp/yajl.rb" on line 1, is calling require("yajl/yajl"), but no file named "yajl.rb", "yalj.so", "yalj.dll", etc., can be found, because it's a relative path, and there is no path containing a "yalj" SUB-directory in the $LOAD_PATH array, containing a file (of any valid extension that require() can load,) named "yalj".

IF you simply copied the "yalj" directory into the SketchUp "plugins" directory, and typed require("yajl/yajl") in the SketchUp Ruby Console, it would be found. (Not to say that it would work, because it itself may have other file dependencies, such as Standard Ruby library files, which requires you to have a full Ruby installation, AND push it's library paths into the $LOAD_PATH array.)

Understanding require() and load(), and how they use (or not,) the $LOAD_PATH (aka $:) and $LOADED_FEATURES (aka $") arrays, is Ruby 101 week 1.

Click on the link in my signature, and follow the advice to collect docs. And just below my Newbie Guide, I posted the old "Pick-Axe" book. Required reading.

Dan Rathbun

@myhand said:

I can also not call them as you would call a static method, i.e. JSON.escape().

OK Example:

Assume your "plugins" dir, has sub-dir "myhand", which has a sub-dir "lib", which contains a file "json.rb"

I recommend having your nested dir names match your nested namespaces, so you can remember the relative paths when it comes time to type a require expression.

module MyHand
  module Lib
    module JSON # mixin module

      def escape(value)
        ret = ""
        value.split("").each do |c|
          if (/["\\\/\b\f\n\r\t]/.match( c ) )
            ret << '\\' << c
          else
            ret << c
          end
        end
        return ret
      end # escape()

    end # module JSON
  end # module Lib
end # module MyHand

.. and in a plugin:

module Author
  module NiftyPlugin
 
    # make sure the lib file is loaded
    require("myhand/lib/json.rb")
 
    # mix in the library module, as a nested module;
    module JSON
      #
      # bring in public mixin instance methods
      # as public module methods in THIS module;
      #
      extend(MyHand;;Lib;;JSON)
      #
    end # module
 
    # use it calling a module method;
    str = JSON.escape("\tHello World!\n")

    puts(str)
 
  end # module NiftyPlugin
end # module Author

OR ... using module_function() like the Math module does ...

module MyHand
  module Lib
    module JSON # mixin module

      module_function()

      def escape(value)
        ret = ""
        value.split("").each do |c|
          if (/["\\\/\b\f\n\r\t]/.match( c ) )
            ret << '\\' << c
          else
            ret << c
          end
        end
        return ret
      end # escape()

    end # module JSON
  end # module Lib
end # module MyHand

.. and in a plugin:

module Author
  module NiftyPlugin
 
    # make sure the lib file is loaded
    require("myhand/lib/json.rb")
 
    # Use a library module function, via a
    # local constant aliasing the library;
    JSON = MyHand;;Lib;;JSON
 
    # use it calling a module method;
    str = JSON.escape("\tHello World!\n")

    puts(str)
 
  end # module NiftyPlugin
end # module Author

OricAtmos

Dan already explained more than I could have. He also wrote a nice ruby script to include additional folders in $LOAD_PATH. Read all about it here: http://sketchucation.com/forums/viewtopic.php?f=180&t=29412&p=342471&hilit=!loadpaths

Myhand

@dan rathbun said:

@myhand said:

I tried to get this to work but keep getting errors when trying to include the library:
require 'yajl'
If you do not specify some kind of filepath, then the file must be in one of the directories listed in the $LOAD_PATH array.

Ruby's require() first checks to see if the argument resolves to an absolute path, and if so, checks to see if the file exists, and if true, attempts to load it.

Secondly, it checks to see if the argument is a relative path (incl. no path at all,) and if so, require() then iterates the $LOAD_PATH array prepending the base paths in front of your relative path. If it finds match, it loads the file, IF such a path is NOT ALREADY present in the $LOADED_FEATURES (aka $") array.

Thank you Dan for the detailed explanation and links to starter guides. I will review and try this when I get home tonight. One interesting point in this case though is that the only file called yajl.* is c:/temp/yajl.rb which is also the file that contains the

require 'yajl/yajl'

line. There is a a subdirectory called yajl, but no files named "yajl.rb", "yajl.so" or "yajl.dll" anywhere else in the yajl library distribution... So cannot see how adding the "c:/temp" path to $LOAD_PATH will find the file as it does not appear to exist.

Will let you know.

OricAtmos

@myhand said:

There is a a subdirectory called yajl, but no files named "yajl.rb", "yajl.so" or "yajl.dll" anywhere else in the yajl library distribution... So cannot see how adding the "c:/temp" path to $LOAD_PATH will find the file as it does not appear to exist.

Sounds like you don't have the complete library.
This is a list of files I have in my Yajl folder:


[...]\rubylibs\yajl.rb
[...]\rubylibs\yajl\1.8\yajl.so
[...]\rubylibs\yajl\1.9\yajl.so
[...]\rubylibs\yajl\bzip2.rb
[...]\rubylibs\yajl\bzip2\stream_reader.rb
[...]\rubylibs\yajl\bzip2\stream_writer.rb
[...]\rubylibs\yajl\deflate.rb
[...]\rubylibs\yajl\deflate\stream_reader.rb
[...]\rubylibs\yajl\deflate\stream_writer.rb
[...]\rubylibs\yajl\gzip.rb
[...]\rubylibs\yajl\gzip\stream_reader.rb
[...]\rubylibs\yajl\gzip\stream_writer.rb
[...]\rubylibs\yajl\http_stream.rb
[...]\rubylibs\yajl\json_gem.rb
[...]\rubylibs\yajl\json_gem\encoding.rb
[...]\rubylibs\yajl\json_gem\parsing.rb
[...]\rubylibs\yajl\version.rb
[...]\rubylibs\yajl\yajl.rb

Myhand

@oricatmos said:

@myhand said:

There is a a subdirectory called yajl, but no files named "yajl.rb", "yajl.so" or "yajl.dll" anywhere else in the yajl library distribution... So cannot see how adding the "c:/temp" path to $LOAD_PATH will find the file as it does not appear to exist.

Sounds like you don't have the complete library.
This is a list of files I have in my Yajl folder:

Yes OricAtmos, I think you are right. I took my version from here

https://rubygems.org/gems/yajl-ruby.

As I cannot install the gem in Sketchup I unzipped the gem file and took the files from the lib directory. This gives me the following files which is clearly not complete.


C;\Temp\lib\yajl
C;\Temp\lib\yajl.rb
C;\Temp\lib\yajl\bzip2
C;\Temp\lib\yajl\bzip2.rb
C;\Temp\lib\yajl\deflate
C;\Temp\lib\yajl\deflate.rb
C;\Temp\lib\yajl\gzip
C;\Temp\lib\yajl\gzip.rb
C;\Temp\lib\yajl\http_stream.rb
C;\Temp\lib\yajl\json_gem
C;\Temp\lib\yajl\json_gem.rb
C;\Temp\lib\yajl\version.rb
C;\Temp\lib\yajl\bzip2\stream_reader.rb
C;\Temp\lib\yajl\bzip2\stream_writer.rb
C;\Temp\lib\yajl\deflate\stream_reader.rb
C;\Temp\lib\yajl\deflate\stream_writer.rb
C;\Temp\lib\yajl\gzip\stream_reader.rb
C;\Temp\lib\yajl\gzip\stream_writer.rb
C;\Temp\lib\yajl\json_gem\encoding.rb
C;\Temp\lib\yajl\json_gem\parsing.rb

Where did you get the library from?

Cheers,

myhand

OricAtmos

@myhand said:

Where did you get the library from?

I don't remember where I got the Windows binaries from. Perhaps I still have a bookmark in my web browser at work, but right now I'm at home and can't have a look. But since I have access to our project repository from home I can offer you this:

https://dl.dropbox.com/u/5140762/yajl-ruby-windows.rar

Myhand

@oricatmos said:

@myhand said:

Where did you get the library from?

I don't remember where I got the Windows binaries from. Perhaps I still have a bookmark in my web browser at work, but right now I'm at home and can't have a look. But since I have access to our project repository from home I can offer you this:

https://dl.dropbox.com/u/5140762/yajl-ruby-windows.rar

Thanks OricAtmos! Will give it a try.

OricAtmos

@myhand said:

Thanks OricAtmos! Will give it a try.

Did you get it to work? Unfortunately I've been unsuccessful in finding out where I got it from.

Myhand

@oricatmos said:

@myhand said:

Thanks OricAtmos! Will give it a try.

Did you get it to work? Unfortunately I've been unsuccessful in finding out where I got it from.

Sorry, I got a bit tied up with trying to solve the MAC bug in my Material_Maintenance Plugin.

Thank you, I have now got it to not throw an error, but my test code does not seem to serialise objects, so will need to read the library docs to see what I am doing wrong

` require 'yajl'

h = {"key1", "val1", "key2", "val2"};

obj = ["Hello", "world", "I am here", ["where", "what", "are", "you"]];

class TestClass

@name = nil;
@adress = nil;
@list = nil;

def initialize(p1, p2)
@name, @adress = p1, p2;
@list = [1,2,3.01,-4.35];
end

end

t = TestClass.new("Richo", "37 Scotland Rd, Buckhurst Hill, IG9 5NP");

obj = ["Hello", "world", t, "I am here", ["where", "what", "are", "you"]];

str = Yajl::Encoder.encode(obj);

puts str;`

Produces


["Hello","world","#<TestClass;0x144d9cc0>","I am here",["where","what","are","you"]]

Instead of the


["Hello","world",{"@list";[1,2,3.01,-4.35],"@name";"Richo","@adress";"37 Scotland Rd, Buckhurst Hill, IG9 5NP"},"I am here",["where","what","are","you"]]

I would expect and which is what my simple JSON Serializer produces.

Myhand

Hi OricAtmos,

I got it to work now thank you. It seems that it can only serialise Hash and Array objects though, which is good enough for most cases I guess. Given the non-trivial install and given the fact that it uses binary libraries, I suspect it will not work on a MAC either without libraries built for that.

I will therefore continue with my simple JSON encoder, which while probably much slower, is simple to install and should work out of the box on a MAC as it is pure RUBY. It can also encode standard objects which can be useful.

Thanks for your help in getting this working though. I will keep it in mind if I ever have to do large data sets.

OricAtmos

All right, I wasn't aware Yajl isn't able to work with arbitrary objects. I was only using it to encode/decode hashes with simple values and arrays.

Myhand

Hi guys,

I need some help. I have run into problems creating a json encoding where a string contains a single quote char. e.g. "Betty's pie shop"

somehow ruby creates a new char which replaces the y and the '. You can see this by executing the following command:

puts "Betty's pie shop"

and I get

Betts pie shops pie shop

if you past this into an editor that supports UTF8 you see that a special char has been added into where the y and ' was. Also note that the last bit of the string is now duplicated.

I can escape it, but this is then not a valid json string as json does not allow you to escape the ' char.