You are all too kind!
I first ran SketchUp (was it v3 or v4) on my old college computer, a pentium III mx, if I recall correctly. It's specs (speed, memory, and storage) were just about on par with the iPhone's. So I'm sure it isn't a hardware limitation, especially if some of the extras are stripped out (If you're in the field, do you really need sketchy edges, or shadows?)
Of course it wouldn't be simple, you'd have to toss the 3button mouse interface (which is perhaps one of the best SketchUp innovations IMHO), and come up with a multi-touch solution to deal with 3D. Also, measurement entry would be very important, but done without a keyboard. That's gotta be hard - I haven't seen it done very well anywhere, yet.
I'd settle for a model markup tool (taking xrok1's suggestion a bit further) - no actual edits, but a simple interface to add notes for later editing.
Actually, I'll settle for whatever the Google Guys come up with. Unless I become a programmer, I'll take what I can get;)