@rspierenburg said:
Is there any chance the wall in the mirror has a reversed face? Could that cause the issue?
Good point! Indeed it was reversed, but fixing it didn't solve the problem. It still looks bad. For now, the solution I found for this, was to render the entire image, then reverse bump map and use render region to render only the mirror, and altough this does not fix the problem I can't even see why it happens to begin with, so it looks like I'm stuck with this solution.