Floating point comparisons of properties

JK
James Kulp
Mon, May 25, 2020 3:10 PM

I ran into a long-standing numeric accuracy bug today with the OpenCPI
expression parser.  I remember someone ran into it a while ago, but then
no one else complained so it was deemed low priority.  Like many other
projects, we use the GNU Multi-precision Library (GMP) library so that
when we do expression math in our property and parameter parsing, we can
use a precision that is the union of the precision of 64 bit
signed/unsigned integers and double precision floats (which have 53 bit
fractions).

We were parsing floating point numbers from text (e.g. property values)
using GMP in some places and strtod in others, which meant that the
actual parsed binary values were not the same.  Since the standard data
types are our user interface, we should always parse floating point
numbers using strtod, and use GMP as intended for intermediate calculations.

I found one worker which was working around this bug, where the default
value of a property was tweaked to avoid the problem (e.g. where a
number like 1.3 would be set to 1.3000001 to avoid the "round trip"
problem).

While direct comparisons between floating point numbers is usually not a
great idea, you do expect that the same string value, when parsed, ends
up being the same floating point value....

This fix should be in release 1.7 which is coming "soon" on gitlab.

I ran into a long-standing numeric accuracy bug today with the OpenCPI expression parser.  I remember someone ran into it a while ago, but then no one else complained so it was deemed low priority.  Like many other projects, we use the GNU Multi-precision Library (GMP) library so that when we do expression math in our property and parameter parsing, we can use a precision that is the union of the precision of 64 bit signed/unsigned integers and double precision floats (which have 53 bit fractions). We were parsing floating point numbers from text (e.g. property values) using GMP in some places and strtod in others, which meant that the actual parsed binary values were not the same.  Since the standard data types are our user interface, we should always parse floating point numbers using strtod, and use GMP as intended for intermediate calculations. I found one worker which was working around this bug, where the default value of a property was tweaked to avoid the problem (e.g. where a number like 1.3 would be set to 1.3000001 to avoid the "round trip" problem). While direct comparisons between floating point numbers is usually not a great idea, you do expect that the same string value, when parsed, ends up being the same floating point value.... This fix should be in release 1.7 which is coming "soon" on gitlab.