Yes, what I was suggesting is.....it doesn't need to be 2 point or parallel to merge....Photoshop recognises where the images overlap and free transforms/skews them to merge into each other automatically.
As panoramas are normally made in a landscape fashion, you may need to rotate your images before processing them. Basically trick photoshop into thinking it's a landscape when it's actually a building. Hope this makes sense.
The above panoramas were made up of around 5-7 single images.