Thanks for the responses, they where quick and helpful but I guess I can't wrap my brain around them. I got one side to work but with a lot of cmd-z and reworking.
I thought I was making all the squares planer on the end, by making a square from the endpoint and then using another endpoint and typing out the dimensions but I guess that doesn't make them right. Because it still will not select just the squares.
EDIT:
I think I might have actually figured it out, I tried the making a big rectangle to use as a drawing surface and that is working now. I believe I was just using the "move" tool wrong. I wasn't select the end points to match up with the end point of the previous square I had place. Hopefully that make sense.