Ruby efficiency for large number of operations

drewmorgan

I made up a small test program that uses the same general flow as my real case, and it produces the same result, so I can post that.. I also generated a simple test file.. This is not the actual file I'm trying to use, but just a test case generated with a script, so its obviously very regular and not very interesting..

Given what you guys are saying, I'm more confident its something wrong with my code, so looking forward to your responses.. Thanks a lot!

The Ruby script from my plugins dir

A test input file

fredo6

Thomthom was right, time is in seconds and the longest is to read the file and generate the Polygon Mesh (4 sec) whereas the face generation takes 0.5 seconds.

This is the output with your 20000 quad faces

[highlight=#ffff00:2915pieu]Reading File and creating Polygon Mesh....
Nfaces = 20000 - Creating Mesh time = 4.291246
Generation of Faces [20000] - Time = 0.517029[/highlight:2915pieu]

Here is the script for that. Note that it inserts a menu item in the Plugins menu
drewmorgan - test_skp_efficiency.rb

Fredo

tt_su

It would probably be possible to squeeze even more performance out of it, but it'd get increasingly more challenging.

If you where able to pre-process all the 3d points first and index them it would save more time. As PolygonMesh is doing a linear search for each point you add.
However, as the data is structured in your sample file this would probably be too slow to do in Ruby. But if you wrote a small Ruby C Extension to read the file and compute the unique set of points needed in C before converting it to Ruby it should take a couple of seconds of the time to create the PolygonMesh.

fredo6

I thought PolygonMesh would do that, based on C code. So doing it in Ruby would not help for performance (which is already quite good).

Fredo

tt_su

Yes, PolygonMesh will merge the points using C, but it's still doing it very inefficiently. It does a linear search each time I received a Point3d object. So when you grow to a large set of points this slows down noticeably.
Which is why you will get best performance out of it if you use add_point first before using add_polygon (with indices).

fredo6

@tt_su said:

Which is why you will get best performance out of it if you use add_point first before using add_polygon (with indices).

Do you mean that add_polygon will accept indexes instead of points? This is not obvious from the doc!
If so, then yes, we can probably improve performance.

Thanks for the tip

Fredo

tt_su

@fredo6 said:

Do you mean that add_polygon will accept indexes instead of points? This is not obvious from the doc!

Correct. I just checked the docs again - I was working from memory - and you're right. It's not obvious, it's only mentioned in add_point.

Using indices you can also control the soft/smooth or hidden properties of the edges bounding the polygons you add. See Entities.fill_from_mesh for more info on how that is controlled.

drewmorgan

Sorry I've not gotten back to you lately - I got pulled off this for a while..

The program you provided is definitely a lot faster and seems to work well, except for in the 20000 face test case, it causes SkecthUp to crash.. It seems to be during the fill_from_mesh call, because it prints out a message to console right before that happens.. But then I get a window that pops up saying "SketchUp Application has stopped working", and it crashes.. It seems to work on some test files I have with a lower number of faces, so maybe its just too many faces for a single polymesh? Does that seem reasonable, or do you think the crash is due to something else?

A couple of observations / things I've learned:

If I make each face be its own group (i.e. make a polymesh INSIDE the loop and fill_from_mesh inside the loop as well) it goes slooooooow. Ideally for my plugin, each one of these would be their own group (of course, what I originally was trying to do is a bit more complicated so each one would be more than a single plate).. I wonder if there's a way to get each one to be a separate group but still be reasonably fast? Based on our previous discussion, I'm guessing once you "fill_from_mesh", it then checks the points that are read in later will all the previous ones, whereas if you just put them all in the polymesh it doesn't do those checks?
If there's more than one implementation of the ruby function in the same directory, SKP will not provide an error message, but will pick one and use that.. This made me think I was losing my mind as I was modifying stuff left and right and not seeing any updates. Turns out when I created the file to post here, I just wrote a subset of my main plugins file to the same dir. All my mods were to my main file, but apparently, SKP was using the (unmodified) implementation in the subset file.. Argh. That took me far longer to figure out than it should have.

tt_su

@drewmorgan said:

The program you provided is definitely a lot faster and seems to work well, except for in the 20000 face test case, it causes SkecthUp to crash.. It seems to be during the fill_from_mesh call, because it prints out a message to console right before that happens.. But then I get a window that pops up saying "SketchUp Application has stopped working", and it crashes.. It seems to work on some test files I have with a lower number of faces, so maybe its just too many faces for a single polymesh? Does that seem reasonable, or do you think the crash is due to something else?

20000 faces shouldn't crash SU, no.
Did you submit the BugSplat? If you did, did you enter some details that I can use to look it up?

tt_su

@drewmorgan said:

If I make each face be its own group (i.e. make a polymesh INSIDE the loop and fill_from_mesh inside the loop as well) it goes slooooooow. Ideally for my plugin, each one of these would be their own group (of course, what I originally was trying to do is a bit more complicated so each one would be more than a single plate).. I wonder if there's a way to get each one to be a separate group but still be reasonably fast?

Got a sample snippet for this? When optimizing for speed it's hard to talk generically. Having a common set of sample code to refer to will help.

@drewmorgan said:

Based on our previous discussion, I'm guessing once you "fill_from_mesh", it then checks the points that are read in later will all the previous ones, whereas if you just put them all in the polymesh it doesn't do those checks?

I'm sorry, but I don't quite follow what you mean here.
fill_from_mesh require that you add the PolygonMesh to an empty Entities collection. It then simply builds the PolygonMesh without doing any more magic.

@drewmorgan said:

If there's more than one implementation of the ruby function in the same directory, SKP will not provide an error message, but will pick one and use that.. This made me think I was losing my mind as I was modifying stuff left and right and not seeing any updates. Turns out when I created the file to post here, I just wrote a subset of my main plugins file to the same dir. All my mods were to my main file, but apparently, SKP was using the (unmodified) implementation in the subset file.. Argh. That took me far longer to figure out than it should have.

That's Ruby for you, it let you override anything. The last one to load will be the one running.

drewmorgan

@tt_su said:

@drewmorgan said:

The program you provided is definitely a lot faster and seems to work well, except for in the 20000 face test case, it causes SkecthUp to crash.. It seems to be during the fill_from_mesh call, because it prints out a message to console right before that happens.. But then I get a window that pops up saying "SketchUp Application has stopped working", and it crashes.. It seems to work on some test files I have with a lower number of faces, so maybe its just too many faces for a single polymesh? Does that seem reasonable, or do you think the crash is due to something else?

20000 faces shouldn't crash SU, no.
Did you submit the BugSplat? If you did, did you enter some details that I can use to look it up?

When I get the crash, I don't get an option to send in the BugSplat report. Do I have to activate that somewhere? I poked around in the preferences, and maybe missed it? I'm only given the option to "Close the program" or "Close the program and look for a solution".. Down in the "details" section, it mentions BugSplat.dll but there's no option to submit it.. Sorry to be dense!

drewmorgan

@tt_su said:

@drewmorgan said:

If I make each face be its own group (i.e. make a polymesh INSIDE the loop and fill_from_mesh inside the loop as well) it goes slooooooow. Ideally for my plugin, each one of these would be their own group (of course, what I originally was trying to do is a bit more complicated so each one would be more than a single plate).. I wonder if there's a way to get each one to be a separate group but still be reasonably fast?

Got a sample snippet for this? When optimizing for speed it's hard to talk generically. Having a common set of sample code to refer to will help.

@drewmorgan said:

Based on our previous discussion, I'm guessing once you "fill_from_mesh", it then checks the points that are read in later will all the previous ones, whereas if you just put them all in the polymesh it doesn't do those checks?

I'm sorry, but I don't quite follow what you mean here.
fill_from_mesh require that you add the PolygonMesh to an empty Entities collection. It then simply builds the PolygonMesh without doing any more magic.

Sorry - not very clear.. What I meant was, in fredo6's posted script, he creates one big PolygonMesh outside the loop over faces, and calls "add_polygon" for each face inside the loop.. Then, after the loop, he creates a new group and calls fill_from_mesh to include all the faces all at once.. That seems to be working fairly quickly. However, if I move the creation of the PolygonMesh inside the loop, add a single face to it, create a new group, and call fill_from_mesh all inside the loop, it crawls and eventually hangs like it was originally..

So, this seems quick:


    model = Sketchup.active_model

    numItemsLine = testFile.gets
    numItems = numItemsLine.split(" ")[2].to_i

    polyMesh = Geom;;PolygonMesh.new(4 * numItems, numItems)
    for i in 0..numItems - 1
      ##...
      polyMesh.add_polygon [face0Vert0, face0Vert1, face0Vert2, face0Vert3] 
    end

    newGroup = model.active_entities.add_group
    newGroup.entities.fill_from_mesh polyMesh, true

but this is slow:


    model = Sketchup.active_model

    numItemsLine = testFile.gets
    numItems = numItemsLine.split(" ")[2].to_i

    for i in 0..numItems - 1
      ##...
      polyMesh = Geom;;PolygonMesh.new(4, 1)
      polyMesh.add_polygon [face0Vert0, face0Vert1, face0Vert2, face0Vert3] 
      newGroup = model.active_entities.add_group
      newGroup.name = "itemNum" + i.to_s
      newGroup.entities.fill_from_mesh polyMesh, true
    end

tt_su

Oh! You're on Windows and you don't get the BugSplat window? :s
Can you send me the script that cause that?

tt_su

@drewmorgan said:

not very clear.. What I meant was, in fredo6's posted script, he creates one big PolygonMesh outside the loop over faces, and calls "add_polygon" for each face inside the loop.. Then, after the loop, he creates a new group and calls fill_from_mesh to include all the faces all at once.. That seems to be working fairly quickly. However, if I move the creation of the PolygonMesh inside the loop, add a single face to it, create a new group, and call fill_from_mesh all inside the loop, it crawls and eventually hangs like it was originally..

I see. I need to look into that.
Have you tried this with using start_operation and using the disable_ui argument?

drewmorgan

@tt_su said:

Oh! You're on Windows and you don't get the BugSplat window? :s
Can you send me the script that cause that?

Actually, I get it with the script that Fredo posted above directly.. I get it with my own script as well, but it is essentially the same as what Fredo posted now that I've mucked with it to be the same general structure..

If I just place Fredo's .rb file in my PLugins directory then select the plugin and load the testFile.txt, I get the crash..

Here's what I get down in the "Problem Details" section of the pop-up window:


Problem signature;
  Problem Event Name;	APPCRASH
  Application Name;	SketchUp.exe
  Application Version;	8.0.11752.0
  Application Timestamp;	4ed14f2c
  Fault Module Name;	BugSplat.dll
  Fault Module Version;	3.1.0.26
  Fault Module Timestamp;	483b5de5
  Exception Code;	c0000005
  Exception Offset;	0000e0c7
  OS Version;	6.1.7601.2.1.0.256.4
  Locale ID;	1033
  Additional Information 1;	43af
  Additional Information 2;	43afc9fd33bc84f320a0c75763285cbb
  Additional Information 3;	4365
  Additional Information 4;	4365e8739e6c211b9e2e8edffa6340e4

Read our privacy statement online;
  http://go.microsoft.com/fwlink/?linkid=104288&clcid=0x0409

If the online privacy statement is not available, please read our privacy statement offline;
  C;\Windows\system32\en-US\erofflps.txt

and here's what I got in the console before the crash occurred:


Reading File and creating Polygon Mesh....
Nfaces = 20000 - Creating Mesh time = 6.147

Oh, I should mention this in case its related to not geting a BugSplat window.. I'm working on a corporate computer, and they are very concerned about security, so every 10 minutes or so, I get a little popup window that says something like "Can not access 0.skb".. I just dismiss it and I'm on my way, but thought I'd mention it in case its somehow related to not being prompted to submit my BugSplat.. I assume its a permissions-type thing (I can't typically write to c:\Program Files (x86), etc)..

I don't think the crash itself is directly related, because as I mentioned, Fredo's script works fine for test files with smaller number of faces, just not for the 20000 one..

tt_su

Can you install and run this tool that will collect more data?

Plugins > SketchUp Diagnostics Tool > Collect Data

su_diagnostics.rbz

tt_su

Oh! "Application Version: 8.0.11752.0" Are you using SU8?

drewmorgan

@tt_su said:

Oh! "Application Version: 8.0.11752.0" Are you using SU8?

Yes, is that bad??

tt_su

That could explain the issues you experience. Any reason you are not using the latest version? Lots of bugs has been fixed since SU8.

drewmorgan

No particular reason I suppose.. I'll try to update to 14 and see if the grouping inside the loop works in the new version.. Meanwhile, thanks so much for all your help - I was able to get the single group created outside the loop to work in a matter of a minute or so for something like 60,000 faces, so I accomplished the main thing I was trying to.. For my particular issue, I'd really like each one to be a separate group so I could name it and be able to click on it and see what the name was, but I'll continue to play with it and update to 14.. Thanks again for your prompt replies and desire to help!