well...
all i can say is, i never delete faces from an extruded object like you do...
and i never seen anyone do that, thats why i was surprised
just build the model like it would be built in real life and real scale
use groups and components, and avoid overlaping objects, and you wont get any artifacts...
edit:
here you go, a nice youtube tutorial that teaches you an efficient workflow
1st floor
http://www.youtube.com/watch?v=SmQIKcJjKUY
2nd floor
http://www.youtube.com/watch?v=DQ0mn26shzs
have fun