Trying to http contact a wrong address always takes 21s

[XorUnison]

As the title says I'm struggling with web requests taking way longer to throw a timeout and move on than they should, 21 seconds. Here's what I'm working with currently:

t=1
puts Time.new
begin
response=Net;;HTTP.start(full_address.host,full_address.port,
;read_timeout=>t,;open_timeout=>t,;ssl_timeout=>t,;continue_timeout=>t,
;write_timeout=>t){|http|http.request(request)}
rescue Exception => e
puts "Rescue block end"
puts e.message
end
puts Time.new

Normally I would think I'd have to only set the open_timeout, as I expect that users might set incorrect variables for full_address. The two time outputs should be no more than a few seconds maximum apart, instead it's this:

2019-02-05 02:32:10 +0100
Rescue block end
execution expired
2019-02-05 02:32:31 +0100

21 seconds, much, much to long for proper user experience.
Now it's not like the t value is completely ignored, just kinda. Setting t to some really high value, something above 21 (or setting no timeout at all) changes the error message, but nothing else, it still takes the exact same amount of time.

2019-02-05 02:41:12 +0100
Rescue block end
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. - connect(2) for "[IP]" port X
2019-02-05 02:41:33 +0100

I've tried multiple ways of rewriting the http request too, all giving me the same dreaded 21 seconds. Anyone have an idea what's going on?