Testing Ruby VALUES in C-extension.

[jolran]

@unknownuser said:

Don't think there is much difference calling a method from a Ruby script or doing the same call from C.

That is very good to know!

@unknownuser said:

What made a difference was that I converted the array of points into C structures and then made all my comparisons and calculations, then converted the result back to Ruby.

I have looked a lot at your C codes, (from Bitbucket). I was wondering about that part.
From an untrained eye it looks rather cumbersome to convert to struct and then back again, isent there a huge cost in doing so ?

What you are saying it's the opposite ? I wouldent mind doing that. It would be a consistent way of dealing with points.
Needless to say, I don't really understand how things work in C-extensions yet..

@unknownuser said:

Are you trying to use 3d points as hash

Hmm, No. I use a Hash as container for points. like this { 0 => pt, 1 => pt etc }
This Hash is then compared/used against arrays of indexes(integers) referring to this Hash.
So I need to lookup the key(index) from value(pt) often when dealing with these arrays in different methods.

The spontanious reaction would be to use an Array instead of Hash for this, but after some "interesting" benchmarks when using array.index(object) I decided to use a Hash.
I havent tried a set yet though, might be better candidate..
I don't know which one is simpler to work with in C.

[tt_su]

@jolran said:

I have looked a lot at your C codes, (from Bitbucket). I was wondering about that part.
From an untrained eye it looks rather cumbersome to convert to struct and then back again, isent there a huge cost in doing so ?

The soft-selection calculation needs to compare each point against every other point, so it's O(N2)!! :s Getting the for each inner iteration, getting the VALUE and then getting the X, Y and Z values and converting them to native values every time leads to a big a big performance hit due to the overhead of invoking the Ruby interpreter.
The "overhead" of doing a pre and post pass to convert the set of 3d points to native C structs is insignificant in comparison. Consider this: I access the Ruby values exactly once! No more than I need to.

That yielded around a 100 times performance increase in my case which was fast enough. If I had needed more improvement I might have done so by using a different data structure to sort my points and doing more efficient lookups - like a K-tree or Oct-tree. But so far it's doing fast enough and there is not need to optimize before it's an issue. (Which is very important to remember by the way!)

And when you optimize, make sure you have done your timing and profiling so you actually know where the bottleneck is. And time your changes so you know you are getting positive results. Occasionally rewrites leads to regressions due to unexpected circumstances.

I can only give generic tip based on my own experience, but ultimately you cannot rely on that to apply to your scenario. Test and measure.

@jolran said:

Hmm, No. I use a Hash as container for points. like this { 0 => pt, 1 => pt etc }
This Hash is then compared/used against arrays of indexes(integers) referring to this Hash.
So I need to lookup the key(index) from value(pt) often when dealing with these arrays in different methods.

I'm not sure you can do that much faster than a hash look up, even Ruby hashes - they are pretty fast for what it's worth. They have a O(1) performance. Maybe I'm not seeing the whole picture of what you are doing, but if you try to simply make a faster C implementation of a hash structure I have my doubts.

@jolran said:

The spontanious reaction would be to use an Array instead of Hash for this, but after some "interesting" benchmarks when using array.index(object) I decided to use a Hash.

That's because arrays are O(n) - in order to look up an item the whole array must be iterated until it find a match. Hashes are constant lookups O(1). I use Hashes a lot in Ruby when I need performance, often they are enough to give me the speed I need.

@jolran said:

I havent tried a set yet though, might be better candidate..

A set wraps a hash - so it should have about the same performance. The difference with a Set is that you only have unique values, while a hash have key-value pairs.
Note that the Ruby StdLib Set is much faster than the old Sketchup::Set class.

@jolran said:

I don't know which one is simpler to work with in C.

If you are diving into Ruby C Extensions I would recommend using our GitHub projects which is in fact a C++ Extension. C++ is much nicer to work with, especially C++11. I spend a lot of time writing pure C extensions early on because Ruby was just plain old C. And I though C++ was even harder - boy way I wrong. Since you are familiar with Ruby and the objective world you will find yourself more familiar with C++ where you can have classes. Constructors and destructor are your friend in C++.

In my latest project I've begun creating a C++ wrapper over the C++ API so in fact a lot of my C++ code looks like Ruby. I might take a chunk of that and make it open source eventually.

For what you are doing right now you might want to look at taking bigger chunks of calculations and doing them in C/C++. If you find that you constantly have to visit Ruby land you might find there is little to gain. Especially if you need to call SketchUp's API as all the methods there are implemented in C++ and the only overhead is the Ruby function call.

If you have some more details on your specific issue I might be able to give more concrete advice.

Btw, if you are unfamiliar with the Big O Notation, here's a couple of links:
http://rob-bell.net/2009/06/a-beginners-guide-to-big-o-notation/
http://bigocheatsheet.com/

[Anton_S]

Interesting topic guys, keep it up

Something tells me SU2015 could have support for plugins in C++. Would be awesome.

[Anton_S]

There is a topic that compares the speed of programming languages to http://www.unlimitednovelty.com/2012/06/ruby-is-faster-than-python-php-and-perl.html

Unfortunately Ruby is ~50 times slower than C++. I think no one would disagree with SketchUp API in C++ and plugins that consisting of cpp files.

[jolran]

@unknownuser said:

Something tells me SU2015 could have support for plugins in C++. Would be awesome.

+1

[jolran]

Awesome answer man! Very rich post

I absorb your recommendations, comments are not necessary

(nice links btw, now I finally understand Log(n)!).

Regarding the Hash lookups. Yes they are fast if using the key. But if looking for the key from a value it's a bit worse. And in my case I see no way around avoiding that lookup.

I'm doing vertex indexing.
http://www.opengl-tutorial.org/intermediate-tutorials/tutorial-9-vbo-indexing/
Where I sometimes have to insert new uniq points, and get the index from a point when sorting the indexed arrays by some algoritm.

@unknownuser said:

If you are diving into Ruby C Extensions I would recommend using our GitHub projects which is in fact a C++ Extension.

I am using the C++ hello_world tutorial as staring point, and expanding on that.
How do you debugg BTW ? It's a bit enoying haiving to quit Sketchup when pasting in .so files. Windows don't allow me to copy and paste over so files that are in use by Sketchup One cannot unload dll's, right ?
There is a debug and release modes(I'm in VS 2010 Ultimate BTW) but I can't launch debugger with SU8 and debug 1.8.
The original project seams to be linked to Sketchup 2013(which I don't have) so I tried to link to SU8.exe to no avail.

What I've seen the recommendation is to use VS 2010 (if windows). Unfortunately VS 2010 lack some of the nice C++11 features, but I guess we can live with that.
A bit difficult to follow some newer C++ tutorials though.

@unknownuser said:

Constructors and destructor are your friend

Yeah, but scary subject. I followed some tutorials here:
http://www.microsoftvirtualacademy.com/Content/ViewContent.aspx?et=5730%26amp;m=5723%26amp;ct=23224
recommending RAII with uniq pointers inside try and catch block. So in our case one could use that to ensure longjumps to Ruby if an Exeption ?
(Way over my head at this stage, but interesting.)
I havent found much info about dealing with objects in C++ and memory handling in C-extensions yet.
I presume the Alloc_n -> xfree trick does not apply in this case ?
Or maybe it's the same in C++ if using a struct and not Class.
A struct must be perfect choice for Point3d in any case ?

Once again. Thanks a lot for that post. Very useful.

[tt_su]

@jolran said:

Regarding the Hash lookups. Yes they are fast if using the key. But if looking for the key from a value it's a bit worse. And in my case I see no way around avoiding that lookup.

Why are you looking for a key by value? Sounds inverted.
Are all your values uniq?
You could keep a second hash with reversed pairs... but then again, it sounds odd and I don't know the full scope of your task. (And I haven't finished reading the rest of your post. :p)

[jolran]

@unknownuser said:

Why are you looking for a key by value? Sounds inverted.

Yeah I may very well have a design-flaw on my hands..

But it is a well-used pattern.

I'm using a Hash with uniq points. And then building different geometries from those points for rendering Open_Gl Triangles and edges.
Normally the ordering of points for face-triangles VS face-edges are different.
So one ends up with 2 arrays of points. 1 for GL_Triangles , 2 for GL_edges.

NOW, Instead of having 3 Containers filled with points. You have only 1 HASH. And 2 Arrays of integers that act as lookups-key to the Hash.
In terms of space this is more lightweight. But The greatest advantage is when for ex Transforming all that geometry. I only have to transform the uniq points.

Using Polygonmesh for this would be better, but one cannot update or purge a polygonmesh.
And I would need to build the 2 arrays anyway, so I have my own class for this purpose.

There are some mechanics for sorting and then I have to lookup the key from a value.
Hope that makes the subject clearer..

[tt_su]

@jolran said:

I am using the C++ hello_world tutorial as staring point, and expanding on that.
How do you debugg BTW ? It's a bit enoying haiving to quit Sketchup when pasting in .so files. Windows don't allow me to copy and paste over so files that are in use by Sketchup One cannot unload dll's, right ?
There is a debug and release modes(I'm in VS 2010 Ultimate BTW) but I can't launch debugger with SU8 and debug 1.8.
The original project seams to be linked to Sketchup 2013(which I don't have) so I tried to link to SU8.exe to no avail.

It is in fact set up to SU2014. When you click run it should launch SketchUp 2014 and a supplementary RB script should load the c extension for you.

@jolran said:

What I've seen the recommendation is to use VS 2010 (if windows). Unfortunately VS 2010 lack some of the nice C++11 features, but I guess we can live with that.
A bit difficult to follow some newer C++ tutorials though.

Indeed. I have VS2013. You can load the project just fine, but if you want to use it as-as you must decline to upgrade it.
If you do choose to upgrade you must modify the Ruby headers as they are not configured to VS2013. That's what I do for my projects now because I like C++11 so much with it's lambdas, smart pointers, ranged for loops and other goodnesses.

@jolran said:

@unknownuser said:

Constructors and destructor are your friend

Yeah, but scary subject. I followed some tutorials here:
http://www.microsoftvirtualacademy.com/Content/ViewContent.aspx?et=5730%26amp;m=5723%26amp;ct=23224
recommending RAII with uniq pointers inside try and catch block. So in our case one could use that to ensure longjumps to Ruby if an Exeption ?
(Way over my head at this stage, but interesting.)

No, you cannot catch Ruby's long jumping error-raising. However, in my testing, if Ruby long jumps out of a function destructors are called - which is a good thing. With C++11 when you have smart pointers that makes memory management a breeze. But if you use plain ol' naked pointers you must take more care. You must either use rb_rescue or re_ensure (which are horrible constructs) or wrap all your memory allocation into classes (which is a good thing to do anyway.)
I recently made a wrapper that converts C++ exceptions into Ruby errors, so I can throw my own custom errors around in my C++ code and have the Ruby layer catch that. Pattern seems to be working quite well, keeps error handling code clean.

@jolran said:

I havent found much info about dealing with objects in C++ and memory handling in C-extensions yet.
I presume the Alloc_n -> xfree trick does not apply in this case ?
Or maybe it's the same in C++ if using a struct and not Class.
A struct must be perfect choice for Point3d in any case ?

From what I've read on best practices people recommend using classes over structs - as a general rule. The reason is that you make accessors to the data members which ensures that you don't expose them directly. Why is that good? Because if you later need to change one of the member to be calculated on demand you just change the accessor method and not every piece of code that points to it.
Alloc_n and xfree are C ways of dealing with memory - which is might kill small bunnies. With C++ you don't need to deal with that.
Exception: if you need to wrap a C data structure in Ruby you need to use xfree to clean up the memory when the Ruby garbage collector collect the object.

@jolran said:

Once again. Thanks a lot for that post. Very useful.

Happy to help! I'm glad more people are looking into more advanced ways of creating extensions.

[tt_su]

@jolran said:

I'm using a Hash with uniq points. And then building different geometries from those points for rendering Open_Gl Triangles and edges.
Normally the ordering of points for face-triangles VS face-edges are different.
So one ends up with 2 arrays of points. 1 for GL_Triangles , 2 for GL_edges.

NOW, Instead of having 3 Containers filled with points. You have only 1 HASH. And 2 Arrays of integers that act as lookups-key to the Hash.

I'd try to wrap that whole calculation in a C extension. C++ extension that is.

@jolran said:

Using Polygonmesh for this would be better, but one cannot update or purge a polygonmesh.

I'm dealing with this right now for my subdivision extension. Working on live update of the mesh when you adjust the creasing. (And any other part of the mesh.) Lots of things to keep in sync - lots of ways to get it wrong.

[jiminy-billy-bob]

This is all very interesting!

[tt_su]

Btw guys, have you heard of DevCamp we're doing in Greece at the end of September?
http://forums.sketchup.com/t/sketchup-developer-event-devcamp-2014/255

If you are able to come we can probably do a C extension session if there is enough interest.

[jiminy-billy-bob]

@tt_su said:

If you are able to come we can probably do a C extension session if there is enough interest.

That would be great!

I'm looking forward to meet you guys there

[jolran]

Wooah! You seam to have gotten far into this C++ business. Well done!

@unknownuser said:

I'd try to wrap that whole calculation in a C extension. C++ extension that is.

I should probably have mentioned that these containers are long lived. The may exist during the whole program.
I do have some more special methods for dealing with other objects(node-editor project) but for this purpouse it would be most intersting to se an example on how you do things in C++ in an isolated case.
In fact, It may be interesting for the whole communitive. (as long as we can use/abuse/expand on it in our projects, that is )

@unknownuser said:

Indeed. I have VS2013. You can load the project just fine, but if you want to use it as-as you must decline to upgrade it.
If you do choose to upgrade you must modify the Ruby headers as they are not configured to VS2013. That's what I do for my projects now because I like C++11 so much with it's lambdas, smart pointers, ranged for loops and other goodnesses.

Ah ok. Well My brother gave me a seat of VS 2010 ULTIMATE. Can't really afford upgrading that..$$$$

@unknownuser said:

It is in fact set up to SU2014. When you click run it should launch SketchUp 2014 and a supplementary RB script should load the c extension for you.

Edit: That doesent happend for me. I get linking errors. I'll look into it more. I wasent aware I could debugg it like that. Sounds a bit more flexible
Ehh. I don't have 2014 either.. Not Pro anyway. Make will work ?

edit: I linked to Sketchup8 which it now seams to find, but I'll keep getting errors.
'SketchUp.exe': Loaded 'C:\Windows\SysWOW64\mscms.dll', Cannot find or open the PDB file
I think I may have linked the RubyStartup "$(SolutionDir)Ruby$(Configuration).rb" file wrong..

For the rest of the information.
Very informative post this is turning out to be!

[jolran]

@unknownuser said:

Btw guys, have you heard of DevCamp we're doing in Greece at the end of September?

Ahhh, Greece. Man, that would have been nice...

[tt_su]

@jolran said:

Ah ok. Well My brother gave me a seat of VS 2010 ULTIMATE. Can't really afford upgrading that..$$$$

Visual Studio Express editions are free. Are you using any features in Ultimate which Express doesn't have? (VS2013 is really nice - I like it so much more than VS2010.)
And using the Test Explorer with the Google Runner extensions is also very nice for setting up unit tests.

@jolran said:

Edit: That doesent happend for me. I get linking errors. I'll look into it more. I wasent aware I could debugg it like that. Sounds a bit more flexible
Ehh. I don't have 2014 either.. Not Pro anyway. Make will work ?

Make will work. Any version of SketchUp that support the RubyStartup argument. (I don't recall when that was added.)

@jolran said:

edit: I linked to Sketchup8 which it now seams to find, but I'll keep getting errors.
'SketchUp.exe': Loaded 'C:\Windows\SysWOW64\mscms.dll', Cannot find or open the PDB file
I think I may have linked the RubyStartup "$(SolutionDir)Ruby$(Configuration).rb" file wrong..

You will get complaints about VS not finding debug symbols for SketchUp. You can ignore that. for mscms.dll, that's a MS component. If you enable the Microsoft Symbols Server it will load most of the PDBs:

[tt_su]

@jolran said:

@unknownuser said:

Btw guys, have you heard of DevCamp we're doing in Greece at the end of September?

Ahhh, Greece. Man, that would have been nice...

You can't make it?

[jolran]

@unknownuser said:

Visual Studio Express editions are free. Are you using any features in Ultimate which Express doesn't have? (VS2013 is really nice - I like it so much more than VS2010.)
And using the Test Explorer with the Google Runner extensions is also very nice for setting up unit tests.

Ah, dang! I just looked it up, it's allowed to do commersial products with the express-version. Well, that's good to know though in any case.
I'm not bound to any features. This all for Sketchup development.
I might try 2013 later. It took quite a while to install 2010 and not without hassle..

@unknownuser said:

Make will work. Any version of SketchUp that support the RubyStartup argument. (I don't recall when that was added.)

Right. I actually had 2013 and 2014(beta-tester version) installed.
So I can link to those.

@unknownuser said:

You will get complaints about VS not finding debug symbols for SketchUp. You can ignore that. for mscms.dll, that's a MS component. If you enable the Microsoft Symbols Server it will load most of the PDBs:

Great! 1 step closer. I still get the PDB complaint but later, but VS seams to be loading a lot of symbols in the background now.
Should I locate the project directory in the "symbol pdb locations" ?
I don't know what I doing..

@unknownuser said:

Ahhh, Greece. Man, that would have been nice...

You can't make it?

Doubt it. This "programming lifestyle" is limiting my budget.

[jolran]

I'm noticing a lot of features lacking in 2010 the more I get into it. So I installed VS 2013.
(It's quite annoying working with the lack of stl initiallizers in 2010 btw)

Indeed I get a conversion promt when starting up Hello_world Project.
So I did convert.

Thomthom, you said one should alter the Ruby headers.
Can I have some more info on that? I assume we are talking about ruby.h file included in external dependancies ? Or do you mean doing changes in the project properties.
In any case I'm a bit lost..

Sorry about the noob questions

[tt_su]

@jolran said:

Thomthom, you said one should alter the Ruby headers.
Can I have some more info on that? I assume we are talking about ruby.h file included in external dependancies ? Or do you mean doing changes in the project properties.
In any case I'm a bit lost..

Yes, ruby.h and/or config.h. I don't recall exactly right now. But if I recall, one of the things where that Ruby define a "inline" macro that conflicts with VS2013.

@jolran said:

Sorry about the noob questions

No worries! I just learned this stuff myself and it's only due to my great colleagues helping me along. I'm still learning.

I'll try to go through the upgrade process again and make notes to what I had to change.