Large Terrain Mesh
-
I'm working on a project that requires me to generate a very large terrain mesh which spans approximately 20 square miles. I have experimented with various methods like taking the raw 3 meter elevation data (DEM) and generating a mesh by importing it into Sketchup but the results were less than satisfactory or Sketchup crashed if I selected too many points. Another method I tried was converting the DEM to contour lines using Global Mapper but again the results were not up to par.
Ultimately I ended up converting the DEM to an XYZ point cloud using Global Mapper and importing that into Sketchup using the Cloud v8 plugin which produced a decent looking terrain mesh. It did require the adjustment of the number of points contained in the XYZ file to obtain a good balance between detail v/s performance but the mesh looked good other than the glitch mentioned in my other thread.
Given the size of the area involved, I am considering the possibility of dividing the area into sections or quadrants to keep it more manageable but it will add additional complexity because I need to keep line of sight in mind. What I am doing is recreating a historic railroad branch line that once existed and will ultimately be using the model as part of a much larger historical research project I am doing. Using scenes, I plan to create videos along the line, so to speak.
What is the best method for handling large terrain meshes while maintaining an acceptable level of detail v/s performance?
The attached image below illustrates the general size and shape of the area involved.
-
@dbwv69 said:
Ultimately I ended up converting the DEM to an XYZ point cloud using Global Mapper and importing that into Sketchup using the Cloud v8 plugin which produced a decent looking terrain mesh.
Can we have a look at it ?
I have not used that method and would like to see the result so far.
I have managed large terrain meshes but doing it by the method of push_pulling. Quite hard.
Thanks
ooops. sorry
did not look at your previous post. -
First of all you need to decide what level of detail you need, and where you want it.
If you're going down at ground level you need more detail, but you'll probably not getting that close to the entire terrain?
There are typically two terrain methods, either using a huge mesh with evenly sized/spaced squares/triangles (DEM) or a terrain built from contour lines, with irregularly sized and spaced triangles (TIN).
Using DEM you need a huge mesh to get any details, while with TIN you can add the details where you need it and make remote areas with just a few tris. If you have a flat area like a football field with DEM it may take many thousands of triangles, while with TIN you could get away with only 2. And at the same time the TIN version would be better defined because it would get precise edges as opposed to the DEM where everything gets kind of blobby. No sharp edges with DEM.
In addition a TIN will show you the terrain in top view, while a DEM will only show a big grid.
You can probably tell tha I'm not a big fan of DEMs
?
If you can get hold of a file with terrain in 3D contour lines (which is what I often get in) you have a good starting point. The you could simplify the contours and use Sandbox From contours to make the terrain.
If not you would have to build it by hand, using Amorph and Artisan or similar.
Attached is a very quick Amorph test on a small part of your terrain. ~450 faces. The control curves are dark cyan, and they can be shaped and detailed just as much (or little) as you want, and the mesh between them will be rebuilt and smoothed. I'd make it finer in regions of interest, and coarser further away from the action. As you can see you get a well defined border between the hillsides and ground (river?)
Amorph is a great tool for terrains, which I wish I had found earlier. I've spent countless hours building terrains with the connect-the-dots technique, sandbox and FlipEdge...
-
You could try to use my Simplify Curves function in my Edge Tools plugin to reduce the number of segments in each curve. I often use that before generating terrains from CAD data.
-
Yes, simplifying contour curves is a must, at least with the ones we get here in Norway from SOSI!
You have a lot of useful tools ThomThom
-
When finished, the terrain will primarily be used to show individual scenes as well as video along the entire length of the historic rail line from a ground level perspective. Imagine sitting in the cab of a locomotive as it travels. That is the perspective I am seeking.
Ideally, everything at ground level, streams, roads,etc... would be at maximum resolution and everything upwards (mountains) would be at a considerably reduced resolution to maximize performance.
So far, the best results I have managed to obtain, which satisfied both requirements of detail and performance, were achived by exporting the raw DEM data to an XYZ point cloud file then importing that into Sketchup and triangulating it there. The mesh was more detailed and much cleaner than anything generated using other methods with the added benefit of being able to trim or fine tune the point cloud prior to triangulation.
Currently, I am experimenting with dividing the area into 3-5 quadrants or irregular sections with some overlapping terrain between each to permit the generation of a seemingly continuous video by carefully synchronizing camera points on a given path.
However, due to the irregular borders of each terrain, I am having difficulty with precisely matching each terrain in 3-D space after import and conversion so that camera paths, roads, etc.. will line up. Additionally, it is important for the terrains to match because I need to overlay topo imagry for building location points and eventually for aerial imagry projection.
In my previous experimentation with generating contours from the raw elevation data, the results were somewhat less than satisfactory. Regardless of what I tried, detail was always lost at the ground level, even when I used contours at 1 meter spacing. Otherwise, I agree that simplifying contour curves is a must.
The Amorph and Artisan plugins look great but I prefer to keep it "free", at least for now anyway.
@bjornkn said:
First of all you need to decide what level of detail you need, and where you want it.
If you're going down at ground level you need more detail, but you'll probably not getting that close to the entire terrain?{SNIP}
-
@bjornkn said:
Yes, simplifying contour curves is a must, at least with the ones we get here in Norway from SOSI!
Aye! I often wish I worked on project located in Denmark or similar places.
(Hvor i landet holder du til?)
-
Out of curiosity I generated an XYZ file at my preferred resolution of 300% above normal sampling and it contains 395983 points. I wonder how long it would take to triangulate and how many triangles the resulting mesh would contain. I have no idea actually, but I am thinking days to triangulate on a P4 and millions of points. Am I even close? For that matter, could Sketchup handle a terrain mesh of that size, especially after it was populated with "stuff"?
Oh how I wish I could use a simgle terrain mesh because multiple terrain meshes are going to be a real pain. LOL
-
That's definitely going to be a pain in the butt to handle in SU. Best would (probably) be to somehow create the footprints of the buildings and everything else and work on those pieces separately.
Speaking of old trains; this is where I was on Saturday:
It goes all along a national park and wild reserve (a flood area of the river Danube)
-
Nice train! It reminds me of one near my home called Cass Scenic Railroad. It was an old logging railroad years ago and was converted into a scenic park for tourists.
On another note, it looks like the decision has been made for me. I tried importing the full XYZ file into Sketchup and it did not like it at all. I imagine that triangulation would crash it for sure.
So, it looks like I will be splitting the area up into separate meshes afterall. Dividing it up into 5 separate meshes would yeild areas with about 75-80 thousand points each. I am not certain but I think even that might be pushing the limits a bit.
That also leaves the question of how to line up the meshes. This is not going to be easy. LOL
-
I did some further testing to see if I could increase the sample spacing, thus reducing the point counts and resulting triangles contained in the generated terrain mesh while still managing to achieve an acceptable result.
First, I created 5 different XYZ files, each having different sample spacings ranging from 100 to 500 percent, from a test area of approximately 1 square mile.
XYZ Sample Spacing = Points
100% = 193053
200% = 48336
300% = 21518
400% = 12008
500% = 7696Next I generated terrain meshes from the imported point clouds for the 400 and 500 percent sample spacing XYZ files. I already knew that the 300 percent file would have yeilded too many triangles in the resulting terrain mesh so it and the others were not tested.
Triangulation Samples:
12008 Points = 23556 Triangles (Faces)
12008 Points = 35563 Edges7696 Points = 12513 Triangles (Faces)
7696 Points = 19972 EdgesThe 500 percent file yeilded a terrain mesh that was just beyond what I would consider an acceptable result but the 400 percent file yeilded a terrain mesh which appeared to be useable.
Using crude math, I estimate that the XYZ file for the entire area would contain about 180,000 points and if divided up into 5 distinct areas, each would contain about 36,000 points. The number of triangles that would be contained in each resulting terrain mesh is unknown.
While waiting for the test meshes to triangulate, an idea came to me...
I could generate and import a point file for the entire area, then edit or delete the points, thus shaping them into 5 distinct areas. It would solve the problem of keeping each mesh in the same 3-D space.
However, I remain uncertain of how I would be able to also import topo or satellite imagery and keep it properly aligned with each terrain mesh. I suppose I could create a point file for a square area as opposed to an irregular area, then trim the point cloud down after everything is aligned but that would mean a lot of extra overhead. There is also the problem of trimming the projected texture images down while keeping them properly aligned.
-
Also using crude math, that would give you a resolution of 18mx18m each square.
Or 14x14m with 400% and 12008. That is much more than you need in remote, less visible areas, but way too little precision for close-up areas as far as I can tell..
I don't know if this is a paid job or not, but if it is I think you're wasting a lot of time trying to save a few bucks. I can't see how you could possibly make such a large area at an acceptable resolution from a triangulated point cloud. You obviously have access to the full data and CAD software since you can export contour lines and/or point clouds?
With a TIN you can also easily align a topo orthophoto, because it is actually possible to see the terrain from above. It would also be very easy to make the terrain rectangular. And with Amorph you could output point clouds for the detail areas only and generate accurate meshes that passes through every one of those points. Great for building sites etc too.
How are you going to handle vegetation, buildings, poles, masts, people, cars(?), tracks + the train itself?
SU is not gonna like 300-500k faces + all the props and objects I'm afraid.
Are you going to render this inside SU, or in some external program?@thomthom: I'm in Kristiansand (just updated my profile too )
Just as unflat here as in Trondheim -
Unfortunately, this isn't a paid project. It is part of a much larger ongoing personal research project I am conducting. Think of it as a means to visualize the things I discover.
As for saving a few bucks, I'm not trying to be stingy, not by any means. It's simply a matter of money. In these tough economic times, I am sure you can understand. Ordinarily, I would not hesitate to spend whatever is necessary in order to achieve my goal but in this case, I am forced to seek alternative and sometimes even creative solutions. It is by no means easy but I like the challenge.
I completely agree about what I have come to call "dual resolution" and have been seeking that very thing, without much success I am afraid. If there were an easy way to keep high resolution at the lower elevations while greatly reducing the resolution at the higher elevations, I would love to do exactly that.
You are correct about my having direct access to high resolution data but Sketchup is the only CAD-like application I am using. I use Global Mapper GIS software to export the raw data into a variety of forms including DEM, DXF, contours, etc... then import that into Sketchup.
The data itself is 3 meter resolution Lidar data provided by the USGS and is nothing short of beautiful when rendered at maximum resolution but it is far more than Sketchup can handle so I am doing my best to find a decent compromise.
My original approach was to use a TIN generated by importing the DEM directly into Sketchup with an accompanying topo overlay projected as a texture. This worked great until I tried to scale things up.
Here is a sample image of a small terrain mesh generated by importing a high resolution DXF mesh file into Sketchup. As you can see, the resolution is superb, but working with the mesh alone was just about all Sketchup could handle.
@bjornkn said:
Also using crude math, that would give you a resolution of 18mx18m each square.
Or 14x14m with 400% and 12008. That is much more than you need in remote, less visible areas, but way too little precision for close-up areas as far as I can tell..
I don't know if this is a paid job or not, but if it is I think you're wasting a lot of time trying to save a few bucks. I can't see how you could possibly make such a large area at an acceptable resolution from a triangulated point cloud. You obviously have access to the full data and CAD software since you can export contour lines and/or point clouds?
With a TIN you can also easily align a topo orthophoto, because it is actually possible to see the terrain from above. It would also be very easy to make the terrain rectangular. And with Amorph you could output point clouds for the detail areas only and generate accurate meshes that passes through every one of those points. Great for building sites etc too.
How are you going to handle vegetation, buildings, poles, masts, people, cars(?), tracks + the train itself?
SU is not gonna like 300-500k faces + all the props and objects I'm afraid.
Are you going to render this inside SU, or in some external program?@thomthom: I'm in Kristiansand (just updated my profile too )
Just as unflat here as in Trondheim -
I'm officially abandoning point clouds.
I can't believe I didn't see it before but Global Mapper gives me the ability to export DXF mesh files, as mentioned previously. As it turns out, I can also change the resolution of the exported mesh files on the X and Y axis to make the exported DXF mesh file more Sketchup friendly.
It doesn't solve the "dual resolution" problem but it yeilds the same or better results than the point clouds did, and all without the hassle of converting DEM's to XYZ files then importing them as point clouds and waiting for triangulation to finish. Overall, the process is 100x faster.
I'll be the first to admit that I am by no means a Sketchup expert, especially where terrain meshes are concerned, but I am learning fast, the hard way.
-
While reading over everything again, I just realized that we may have been using a term in very different contexts and with different meanings.
Several times I mentioned DEM's only in my case, I meant Digital Elevation Models (GIS data). From reading everything, it appears that the term DEM has a different meaning in Sketchup terms. Previously, I had always assumed that anything (terrain mesh) with a network of triangles was a TIN but it looks like I was mistaken. Can someone set me straight on this one?
-
DEM and TIN are not SketchUp specific terms, but pretty general.
A DEM is a grid with evenly spaced x and y coordinates. For each intersection there is an altitude value. Which means than when viewed from above with no perspective any terrain will look exactly the same, just like a flat grid. It would be impossible to see if it was from the Alps or Holland. No roads, rivers or anything will be visible.
It uses the same grid everywhere, and wastes a lot of unnecessary vertices on flat or even large areas. Even flat water may use thousands of vertices.
A TIN, which means 'Triangulated Irregular Network' is different, because it only should place vertices where there needs to be one, like along a road for instance. Typically they are often made from contour lines stitched together.
The advantage is that you can see from above what the landscape looks like, where the roads are etc.
You can also get sharply defined roads etc, unlike with DEMs where everything is soft. A road in a DEM with a resolution of 6x6m will be pretty much invisible even when seen in perspective.
The main advantage of a TIN, particularly in SU, is that it can be made to cover a huge area at very little cost face-wise, because you can make it as coarse you like in distantly viewed areas, or as detailed as you like in ROIs that will be seen closeup.
Polygon reducers, like those present in Amorph and Artisan will be able to remove a lot of unnecessary faces in DEMs in areas of low frequency, like flat areas etc, but you will still not get crisp edges along roads etc. With orthophotos applied you will often get water climbing up on the terrain around lakes, rivers etc (just take a close look at lakes in Google Earth!) because the resolution is too coarse, and the waterline doesn't follow the square grid. With a TIN you will get a completely flat water plane which could be perfectly textured.
You can get good results with SU and the "From contours" tool in Sandbox, but be prepared to do lots of cleanup.
It isn't that bad actually. The technique I've used for years has been to draw coarse contour lines on top of the imported ones, adding points wherever there's a change of direction etc, and often skipping several altitudes. And then skin a selection of such contour lines with Sandbox, cleanup by flipping edges to remove flat areas, and then explode to merge with the rest of the topo.
The attached terrain was completely handmade using this technique. It took maybe 4 hours and is about 12000 faces?
I tried a small part of that terrain using the original contour lines, but simplified, and it ended up with 40000+ faces and wasn't looking any better. Today I would've used Amorph.
Attached also a version with orthophoto applied. As you can see the entire hill is covered with trees, so even that coarse topo was really overkill anyway. But today there very few trees left there - it's covered with houses, school etc. Unfortunately that job was canceled before I even got to start to build that school
-
I am starting to believe that Sketchup simply doesn't have the horsepower to do what I need to do.
Does anyone know of any decent (free) 3D Mesh poly reduction programs? I heard that Blender can do it but I'll be darned if I can make heads or tales of it's odd user interface. I also tried Meshlab but it crashes at the drop of a hat.
Then there is the problem of converting DXF or other files to formats that the various 3D programs can import. You'd think the 3D software world was ran by monkeys with all the variations of files and interfaces that are out there.
This is starting to give me a headache the size of Texas.
-
@dbwv69 said:
I am starting to believe that Sketchup simply doesn't have the horsepower to do what I need to do.
Does anyone know of any decent (free) 3D Mesh poly reduction programs? I heard that Blender can do it but I'll be darned if I can make heads or tales of it's odd user interface. I also tried Meshlab but it crashes at the drop of a hat.
Then there is the problem of converting DXF or other files to formats that the various 3D programs can import. You'd think the 3D software world was ran by monkeys with all the variations of files and interfaces that are out there.
This is starting to give me a headache the size of Texas.
Can you post your model. It will probably not fit the 3d ware house limit and probably not the Google docs but possibly one of the free ftp sites or post a section. Thinking you may not have not spent enough time with MeshLab? Check drop box it gives a large file size upload
-
@dbwv69 said:
I am starting to believe that Sketchup simply doesn't have the horsepower to do what I need to do.
It's not so much lack of horsepower as getting used to driving with a manual shift
But if you want to drive at top gear everywhere you'll not gonna make it work for you.
If you absolutely need to have a 3x3m res in all of your terrain then SU is not the program for the job. 5-6 million polys can probably easily be handled with Blender, and so can most other non-free 3D programs, like LightWave, which is what I'm using when I need thousands of instanced 3D trees, rocks, grass and lots of polys.
In SU you have to "adapt to the terrain" and keep your polycount as low as possible wherever there's no need for lots of detail.
The SU community is so spoilt by all the great free plugins available, as well as lots of affordable ones. Sometimes you can't do without them, but sometimes you can spend a few days/weeks working hard instead of spending $39/$149.@unknownuser said:
Does anyone know of any decent (free) 3D Mesh poly reduction programs? I heard that Blender can do it but I'll be darned if I can make heads or tales of it's odd user interface. I also tried Meshlab but it crashes at the drop of a hat.
Then there is the problem of converting DXF or other files to formats that the various 3D programs can import. You'd think the 3D software world was ran by monkeys with all the variations of files and interfaces that are out there.
This is starting to give me a headache the size of Texas.
As advertised on this page currently there is http://www.mootools.com/plugins/us/sketchucation/index.htm which also comes with a poly reducer, the Polygon Cruncher. It works great, with lots of control, but again it is not free.
-
@bjornkn said:
It's not so much lack of horsepower as getting used to driving with a manual shift
But if you want to drive at top gear everywhere you'll not gonna make it work for you.
If you absolutely need to have a 3x3m res in all of your terrain then SU is not the program for the job. 5-6 million polys can probably easily be handled with Blender, and so can most other non-free 3D programs, like LightWave, which is what I'm using when I need thousands of instanced 3D trees, rocks, grass and lots of polys.
In SU you have to "adapt to the terrain" and keep your polycount as low as possible wherever there's no need for lots of detail.
The SU community is so spoilt by all the great free plugins available, as well as lots of affordable ones. Sometimes you can't do without them, but sometimes you can spend a few days/weeks working hard instead of spending $39/$149.In my case, it was a matter of getting used to driving an automatic. I could achieve this with any number of other 3D programs, including AutoCAD, but none of them offer the simplicity or ease of use that Sketchup offers. For a model of this nature, the precision such as that offered by AutoCAD simply isn't necessary and would be akin to using a construction crew to build a park bench.
As stated previously, I do not need high resolution in the entire model and am trying to achchieve some level of compromise between high resolution in the valleys and low resolution in the mountains.
What I am trying to do is possible and one way or another, I will achieve my goal, even if it means learning Ruby and writing my own plugin. I am a programmer ya know, just not in Ruby.
@bjornkn said:
As advertised on this page currently there is {SNIP}
I can appreciate that you want to make a few bucks but I've already made it very clear that I am not interested in buying a plugin at this time. Using pretense to lead into a sales pitch isn't going to change my mind. Besides, I am not fully convinced that your plugin will do exactly what I need to do.
Finding free solutions may not be easy, especially when they are buried under a mountain of advertisements or misleading "free" solutions that aren't really free but don't tell you that until you download and try them, but truly free solutions are out there and I fully intend to find them.
Advertisement