A
number literal is a sequence of mostly digits consisting of one
or more of the following characters: an integer, a decimal point,
a fractional segment, an exponent indicator (“e” or “E”), and an
optionally signed exponent value. These are all examples of literal
numbers:
-12
1.5362
0.875
5.56e-2
1.234E10
It is possible to omit either the integer or fractional segment
of a literal number, but not both. In addition, within the fractional
segment, you can omit either the decimal point or the exponent value,
but not both.
All number literals are internally converted to Institute of
Electrical and Electronics Engineers (IEEE) 64-bit binary values.
However, IEEE values can only represent a finite quantity of numbers,
so certain values do not have a representation as a binary fraction.
This is similar to the fact that certain values, such as 1/3, do
not have a precise representation as a decimal fraction (the decimal
value would need an infinite number of decimal places to be entirely
accurate).
The values that do not have a binary fraction equivalent are
generally number literals with more than 16 significant digits prior
to their exponent. FormCalc rounds these values to the nearest representable
IEEE 64-bit value in accordance with the IEEE standard. For example,
the value:
123456789.012345678
rounds to the (nearest) value:
123456789.01234567
However, in a second example, the number literal:
99999999999999999
rounds to the (nearest) value:
100000000000000000
This behavior can sometimes lead to surprising results. FormCalc
provides a function, Round,
which returns a given number rounded to a given number of decimal
places. When the given number is exactly halfway between two representable
numbers, it is rounded away from zero. That is, the number is rounded up
if positive and down if negative. In the following example:
Round(0.124, 2)
returns 0.12,
and
Round(.125, 2)
returns 0.13.
Given this convention, one might expect that:
Round(0.045, 2)
returns 0.05.
However, the IEEE 754 standard dictates that the number literal 0.045 be approximated
to 0.0449999999999999. This approximation is closer
to 0.04 than to 0.05. Therefore,
Round(0.045, 2)
returns 0.04.
This also conforms to the IEEE 754 standard.
IEEE 64-bit values support representations like NaN (not a number),
+Inf (positive infinity), and -Inf (negative infinity). FormCalc
does not support these, and expressions that evaluate to NaN, +Inf,
or -Inf result in an error exception, which passes to the remainder
of the expression.
