i see those splotches in the floor and maybe a bit in the dark panels in the middle too.. try to render a separate raw gi pass, to better see how gi is working overall..
assuming that everything else is fine in your irradiance map AND light cache setup.. i would try to reduce the color treshold value and see if that help..