Even changing the Toolbar icon size will affect the viewport.
I always ensure that I only have one row of small icons in my view - I'm kind of stuck with it due to the number of projects set up to match SU output and render output.