OK, so I expected to find a lot of answers on the internet for how to find the center of projection (or attention) for a photo, but surprisingly I didn’t find any techniques. Ok there are techniques that look at peoples eyes to see if they have been composed into a photo; this work through finding the center base on analysis of the eyes - the bad eyes will project a different center than the rest; but not really relevant for PhotoMatch.
Well two things came to mind that I thought would be useful for this. First is that there isn’t just one horizon line, but rather three (for cube shaped object – 3 90deg axis). Second is that each horizon although it goes off infinitely in both directions, kind of has a center. This center is the point along that horizon, where a vertical line doesn’t tilt in or out regardless of tilt of camera. It is perpendicular to the horizon and goes through the third vanishing point (the one not forming this horizon). Ok, a third thing was that as you tilt your camera up, your verticals start to converge at the top, and your horizon moves lower on your photo. the question was how to translate the amount of vertical vanishing to a camera tilt amount; this would seem to vary based also on the FOV of the lens.
So I drew this line for the three horizons, and wouldn’t you know it, they all cross at (or very close to) the center of the original photo. You can crop the photo all you want, but this point won’t change. Of course you need to have content in the photo to define the vanishing points.
The attached photo (pic 3) shows me drawing the lines perpendicular to the horizons through the opposing vanishing point. This particular test was off about 3% from the true center, but keep in mind, this error may be the camera’s sensor, they are always off to some extent, although 1% would be more typical. The other flaw is that I may not have drawn my lines correctly do to use error, or other lens distortion. But I think its close enough to provide good results. Had this photo been cropped off center, I would now open in Photoshop and expand the canvas with extra white space on the sides making sure to re-center on the newly found center of projection.
So it seems this works great when you have the camera tilted up or down (where there is a vertical vanishing point), but when you are truly or close to being level, your vertical vanishing point is so far off that verticals appear parallel to each other. Well, that doesn’t let me get my center of projection. I know it is on the x-y horizon so I can resolve vertical cropping, but I won’t be able to correct for cropping to the sides of the photo in this case. This doesn't seem to be an issue for PhotoMatch though. If you don't have converging verticals, then you don't have to worry about cropping to the sides. It is probably a good idea however in this case to rotate the photo so the horizon is level. This way no-matter where the center was across the horizon, it is still vertically in the center of photo.
Happy un-cropping. I'm hoping to not find the need for this in SU7, and also hopping for mostly automated calibration as well. Maybe also option for telemetry based solution to work with non-square and non-adjacent geometry. (but I'm just guessing and wishing)
Feel free to post problem images, and let me or others take a stab at finding the center.