Length woes

[g2ktcf]

Al,

I do remember reading something about "within a tolerance" for many of the SU functions. I do not know what the tolerance is but you may be getting two values the the sytem considers EQUAL within a tolerance. Then the "<" would be false as both numbers are Zero within this established tolerance (i thought it was 0.01")

just my quick after lunch thoughts...

Chris

[Didier Bur]

Yes,
In fact, I often had to write my own "within tolerance" method, as well as several other coders I think. Safer...

[g2ktcf]

What is the actual "tolerance" value? Anyone?

[todd burch]

Try Float::EPSILON.

Todd

[g2ktcf]

Okay...that's a REALLY small number.

[g2ktcf]

Okay, I had to play this morning on this.

I took the same L1 and L2 example and varied the value of l2 by a factor of 10 until the L1==L2 statement returned FALSE. It seems that a value of l2=0.001 is the point at which things change. I tried an l2 = 0.0005 and that returned TRUE. This must be handled in the class methods....Right???

[g2ktcf]

It seems the same argument goes for the Zero Vector. Setting either of these with a value less than 0.001" will result in SU "seeing' them as zero even though they print out differently.

[TIG]

"float-tolerance.rb" is by RickW [I think - I hope he doesn't mind me repeating it in its entirety here]

class Float
	def =~(num2,tol=10000)
		num1=(self*tol).to_i
		num2=(num2*tol).to_i
		return true if num1==num2
		return nil
	end
end #class Float

Then you test two floats thus:
float1 =~ float2
It returns false or true
You could adjust the float tol part to suit...

[RickW]

I don't mind as long as there's a link
(http://www.smustard.com/script/FloatTolerance )

[g2ktcf]

I actually found this implied in an example in the length class API. If you use a float, the math comes out like you would expect. The Length class is where this changes (as Al suspected). Notice that the 0.001" value that I got while testing is not mentioned as a limiting factor.

@unknownuser said:

<
The < method is used to see if one length is less than another length.

Syntax
status = length1 < length2
Arguments
length1 - a length value

length2 - a length value

Return Value
status - true if length1 is < length2; false if length1 is not < length2

Comments
For example, if l1 = 1.0.inch and l2 = 1.000001.inch then l1 == l2 so l1 < l2 should return false.