well...
all i can say is, i never delete faces from an extruded object like you do...
and i never seen anyone do that, thats why i was surprised
just build the model like it would be built in real life and real scale
use groups and components, and avoid overlaping objects, and you wont get any artifacts...
edit:
here you go, a nice youtube tutorial that teaches you an efficient workflow
1st floor
Youtube Video
2nd floor
Youtube Video
have fun