Here's the issue, I think.
If you use leader text on a pdf image, the leader arrow points "stick" to the image so selecting all of it and moving is not a problem.
If you add a clipping mask to an image, and then the leader text, the leader arrow points will not "stick" to the image. They will stick to the page location. HOWEVER the way to move them is to move by dragging a leader when you select all of it and perform the move. All of the selected leader text will move as expected along with the image. This is true if you simply write leader text in empty space as well--it sticks to the page unless moved by the leader.
It seems these methods are a little inconsistent and, even if documented, not intuitive. Yet, given that the leader apparently MUST be "stuck" to something, this is one solution. I only wish that adding a clipping mask did not change the behavior.