Vray crashing SU when rendering landscape

[mr_tuel]

First off, sorry for the long post. I'm new to the forum, not all that new to SU, but relatively new to Vray. I have read through the Vray manual, and played with it quite a bit. I Still don't understand all of the settings, but I understand how some of the maps work, etc. I am using the Vray trial (I keep re-installing) because I'm a broke college student.

I use two computers to model. Primarily:

Self Built
Intel Core i7 920 (4 multi-threaded cores) overclocked to 3.4 GHz
6GB RAM
640 GB hard disk
2x Nvidia GeForce9800GTX+ video cards
Windows 7 x64 Home Premium installed
Originally had SU 7.1.6087, then upgraded to 8.0.3117
Vray 1.48.89

When I render on this box, with a layer that has about 40 low poly 'cypress' shrubs, no problems. With 10 very high poly 'live oak' trees, and no other plants, it crashes with an error message about not being able to create an error report because my temp folder is out of memory (it's not). Right before this happens, the FrameBuffer loads. As the FrameBuffer loads, SU uses about 2GB memory, and CPU briefly hits 100%. Note that with both layers on, I get the same result.

I also use a MacBook Pro when I'm at school or otherwise away from my primary computer:

Model# A1260
Early 2008, right before the 'unibody' generation
Intel Core 2 Duo T8300 at 2.4 GHz
4GB (max supported) RAM
500GB hard disk
Nvidia GeForce 8600M GT video card
Windows 7 X64 Ultimate installed
Sketchup 7.1.6087
Vray 1.48.89
When I Render on the laptop, no probs with either the cypress by itself or the oak by itself. With both layers on, Vray and SU become unresponsive and stay that way for as long as 45 minutes (the longest I've waited). I can 'kill' the programs and reopen the files fine with no error messages.

Model Stats:

13,382,520 edges, and I expect to wind up with another 10 million
1,065,847 faces
127,505 Component Instances
97,897 groups
62 component definitions
10 layers
64 materials, though several aren't used in the model

I've been rendering at 800x600 to save time, I will render at higher res for presentation.

I obviously have a couple of different problems. I have tried reinstalling Vray, reinstalling SU, and even upgrading SU, to no avail. I realize I could just as easily use 2D plants, but they don't have nearly the same visual effect. I would really prefer to use my desktop since it can process 4x the image in the same number of clock cycles, usually being about 65-70% faster than the MacBook. If nothing else I'm willing to not render on the MacBook at all. So, my main concern is to get my desktop computer to render my model.

Is my model too complex? Is there a memory issue with the software? Can my problem be fixed with a setting(s) and which one(s)?
Any tips or trick to make this model render? Do you need a copy if the model?

Thanks for reading, and thanks in advance for your help!

[thomthom]

I don't know how things work on OSX - but for windows:
SketchUp is a 32bit application - and 32bit application can address only 2GB of memory - even if you are on a 64bit system and have more. It's simply a limit of 32bit.

Them trees of your, I'm guessing that the leaves are clipmaps? Calculating transparency maps require more resources.

When you do run into memory issues, you can try to render directly to file - to VRImage: http://software.asgvis.com/index.php?option=com_content&view=article&id=16&Itemid=437
(Note that I doing the IR map in a prepass isn't a requirement.)

There could be other things with your settings that cause it to use more memory than necessary. But this can be anything.

Then finally - and this is a USE AT OWN RISK!
Make backup of your SketchUp.exe before attempting:
There is a utility that can make SketchUp LargeAddressAware: http://forums.techpowerup.com/showthread.php?t=112556
When an 32bit application runs on a 64bit OS it can address up to 4GB - double the norm. I've tried this myself to get a render through.

[thomthom]

btw - do you have any displacement in your scene?

[mr_tuel]

Thanks for the replies.

There is displacement for alot of the materials, should I remove/reduce it? The 32 bit memory addressing issue makes sense, though I thought the limit was 4.3GB, since 2^32=~4.3 billion, which in bytes is ~4.3 GB. But Maybe that was only for the OS, and it limits programs to 2GB? In either case I know more about hardware than software anyway.

The trees are pure geometry, not images or maps. Just edges and faces. That is one reason I thought I might be having problems, because they are geometrically complex. Sketchup doesn't appear to like complex geometry very much. I was able to render the scene using Twilight, but that had its own problems when the edge count later hit about 20 million.

Would it do any good (for rendering purposes) to trim out unused layers and materials? I know I probably should anyway, but could that possibly be causing any issues?

I want to point out that at no time am I using OS X. While I was using a Mac mostly for troubleshooting, Windows 7 x64 is the OS on both machines. Now if OS X might give me better results, I might have an excuse for a Mac Pro (or to convert my i7 rig into a Hackintosh).

I don't find that Sketchup is all that ideal for full 3D landscapes that are to be ray-traced and rendered, but it's great for a lot of other modeling, especially conceptual modeling. However, it's the only 3D modeling program I know how to use at the moment, and I have projects with deadlines.

Thanks in advance for any additional help!

[thomthom]

Which of the materials are displaced?
From your sample render I don't see any materials where it's obvious. Maybe bumpmap would do?
Displacement is very memory intensive. And is you use it on large surfaces you force V-Ray to keep it in memory for longer - so you want to subdivide large displaced surfaces.

But - you have a ~1million face model - it might very well be all the geometry that eat your memory.