PDA

View Full Version : !DTOR and !RADEG are incorrect



hcrisp
04-23-2009, 01:32 PM
Message redacted. Reconsidered the problem and needs more investigation...

hcrisp
04-24-2009, 08:36 AM
Ok, here's what I know now:

It would be better to say !DTOR and !RADEG are imprecise. !DTOR and !RADEG are float type, and thus only have float precision. Since there is no double-type !DTOR or !RADEG, here is the list of conversions factors for radians to degrees in order of descending precision (highest to lowest):



1) 180.D / !DPI
2) !RADEG or FLOAT(180.D / !DPI) or DOUBLE(!RADEG)
3) 180. / !PI

; (substitute 1./!DTOR for !RADEG, since they are reciprocally equivalent)


Here's the proof:


print, string(180.D/!DPI, format='(F30.20)')
; 57.29577951308232300000
print, string(!RADEG, format='(F30.20)')
; 57.29578018188476600000
print, string(FLOAT(180.D/!DPI), format='(F30.20)')
; 57.29578018188476600000
print, string(DOUBLE(!RADEG), format='(F30.20)')
; 57.29578018188476600000
print, string(180./!PI, format='(F30.20)')
; 57.29577636718750000000


Binary display of float conversion factors:


; !RADEG
01000010 01100101 00101110 11100001
; FLOAT(180.D/!DPI)
01000010 01100101 00101110 11100001
; 180./!PI:
01000010 01100101 00101110 11100000


In summary, use 180.D / !DPI for highest precision, and avoid 180. / !PI except in cases where you can afford the round-off!