PDA

View Full Version : most negative integer

hcrisp
05-20-2009, 01:55 PM
I noticed a strange thing about the INT datatype that was causing an error in a function I wrote.

All integers in the integer range are typed integer -- except one. The most negative integer is assumed to be a LONG, unless you explicitly type it as an INT.

info, 1024
; <Expression> INT = 1024
info, -1024
; <Expression> INT = -1024
info, -32768
; <Expression> LONG = -32768
info, FIX(-32768)
; <Expression> INT = -32768

I am just curious. Was this by design? Is this the only number that behaves this way (where it assumes the wider type unless down-typed)?

donb
05-21-2009, 04:36 PM
Yes, the results you see are very expected and very much by design.

PV-WAVE automatically chooses the variable type based the value. In the case of -32768, that value crosses the boundry of what can be stored in a 16 bit signed INTEGER, so PV-WAVE propogates that to a LONG. The reason you can print the FIX of that same number is because the negative sets the sign bit and still fits into a 16 bit INTEGER.

We're really talking about bit patterns in memory and how those are interpreted. If you create a variable with 15 1s, 16 1s, and then 24 1s, you end up with:

info, '7fff'x
;<Expression> INT = 32767
info, 'ffff'x
;<Expression> LONG = 65535
val='ffff'x
print, fix(val)
; -1
val2='ffffff'x
print, fix(val2)
; -1
print, val2
; 16777215

Notice that the 'fix' of both 16 and 24 1s give you same result - even though the value is a LONG, it takes the bottom bits that fit into an INTEGER (all 1s in both cases) and it prints the interpreted value (-1).

To answer the last question, yes, this is a rather unique bit pattern and boundry condition that results in this type of behavior. If you were on a 64-bit machine and we had a 32-bit data type, you would see a similar condition and results as you cross the 32- vs. 64-bit boundry.

Hope this helps,

Don B.

hcrisp
05-22-2009, 12:19 PM
Maybe another way to say it is, the number is cast first, and then the negative operation is performed.

Because positive 32768 is a LONG, not an INT, it is cast as a LONG. IF you try to FIX() it first, then it wraps to the negative end, and no negative operation is needed. If one is performed, it wraps twice, and the result is still a negative INT.

WAVE> info, 32768
; <Expression> LONG = 32768
WAVE> info, -1 * (32768)
; <Expression> LONG = -32768
WAVE> info, FIX(32768)
; <Expression> INT = -32768
WAVE> info, -1 * FIX(32768)
; <Expression> INT = -32768

As you mentioned previously, you cannot do this on LONG, because WAVE will not promote it to FLOAT. Instead, it wraps to negative, and then the negative makes it wrap twice:

WAVE> info, -2147483648
<Expression> LONG = -2147483648
WAVE> info, 2147483648
<Expression> LONG = -2147483648
WAVE> info, -1 * 2147483648
<Expression> LONG = -2147483648

You can't do this with BYTE because the interpreter won't let you. However you can cast an INT down to BYTE, and it will wrap:

WAVE> info, 256B

info, 256
^
%%%Byte constant must be less than 256.
WAVE> info, byte(256)
<Expression> BYTE = 0

All very interesting cases, and it helps to know what the interpreter is doing.