Problem solved!
Seems like when Shadow Radius is >0 the physical size of the spotlight component affects shadows. By increasing the size of the component itself I was able to obtain the shadows that I were looking for.