We've found two instances of extra slow rendering with vector, and I'll point out one and we're working on fixing the other.
If your view in LayOut only has a tiny bit of geometry, but yet there's a large (geographic area of model not shown), you do render all of the model and it can be slow. Dave R helped us find this, and we're looking at how this happens but no workaround other than to have you move items that aren't geolocated closer to each other, or possibly do them in separate models and include all of them in LayOut separately.
An edge issue in vector, which we're testing fixes for, that can cause some models. A user in our Google support forums provided us a test file which we were able to analyze. We'll keep you posted on how this goes and if it helps.
Unlike raster images, the vectors aren't cached and are re-rendered, so even without these issues, you still may get slow rendering. We are currently working on making updates to models be less painful when updated in LayOut.
b