axis-c-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fred Preston (JIRA)" <axis-c-...@ws.apache.org>
Subject [jira] Commented: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
Date Wed, 23 Nov 2005 12:45:36 GMT
    [ http://issues.apache.org/jira/browse/AXISCPP-838?page=comments#action_12358351 ] 

Fred Preston commented on AXISCPP-838:
--------------------------------------

Precision problems with floats
------------------------------------------
Serialisation
-----------------
Before trying to resolve anything, I though I would do some investigation into floats... 
I first created lots of float values around the 555.555 problem area to see how this number
was being represented...

Code                                                                       "%f" Produced 
   "%.7g" Produced  "%.8g" Produced
xsd__float      myFloat00 = 555.5 + 0.0001;           555.500122           555.5001      
    555.50012
xsd__float      myFloat01 = 555.55 + 0.0001;         555.550110           555.5501       
   555.55011
xsd__float      myFloat02 = 555.555;                       555.554993           555.555  
           555.55499
xsd__float      myFloat020 = 555.555 + 0.1;           555.655029           555.655       
      555.65503
xsd__float      myFloat021 = 555.555 + 0.01;         555.565002           555.565        
     555.565
xsd__float      myFloat022 = 555.555 + 0.001;       555.556030           555.556         
   555.55603
xsd__float      myFloat023 = 555.555 + 0.0001;    555.555115           555.5551          
555.55511
xsd__float      myFloat024 = 555.555 + 0.00001;  555.554993           555.555            
555.55499
xsd__float      myFloat03 = 555.5555 + 0.0001;    555.555603           555.5556          
555.5556
xsd__float      myFloat04 = 555.55555 + 0.0001;  555.555664           555.5557           555.55566
xsd__float      myFloat05 = 555.556 + 0.0001;       555.556091           555.5561        
  555.55609
xsd__float      myFloat06 = -555.5 - 0.0001;          -555.500122         -555.5001      
    -555.50012
xsd__float      myFloat07 = -555.55 - 0.0001;        -555.550110         -555.5501       
   -555.55011
xsd__float      myFloat08 = -555.555 - 0.0001;     -555.555115         -555.5551         
  -555.55511
xsd__float      myFloat09 = -555.5555 - 0.0001;   -555.555603         -555.5556          
-555.5556
xsd__float      myFloat10 = -555.55555 - 0.0001; -555.555664         -555.5557           -555.55566
xsd__float      myFloat11 = -555.556 - 0.0001;      -555.556091         -555.5561        
  -555.55609
xsd__float      myFloat12 = 555.5;                             555.500000          555.5 
                 555.5
xsd__float      myFloat13 = 555.55;           (5)           555.55
                                                                         (6)           555.55
                                                                         (7)           555.55
                                                                         (8)           555.54999
xsd__float      myFloat14 = 555.555;         (6)           555.555
                                                                         (7)           555.555
                                                                         (8)           555.55499
xsd__float      myFloat15 = 555.5555;       (7)           555.5555
xsd__float      myFloat16 = 555.55499;     (6)           555.555            <- This has
rounded up rather than truncated.
                                                                          (7)           555.555
           <- This has rounded up rather than truncated.
                                                                         (8)           555.55499
xsd__float      myFloat17 = 555.5499;       (6)           555.55            <- This has
rounded up rather than truncated.
                                                                          (7)           555.5499
                                                                          (8)           555.54993

The best guaranteed precision without any rounding appears to be 5 digits and with rounding
6 (although I would need to confirm this).

Reading through the IEEE numeric standards, types have the following precision:-
float =  sign bit, 8  bit exponent, 23 bit mantissa
double =  sign bit, 11 bit exponent, 52 bit mantissa
long double = sign bit, 15 bit exponent, 64 bit mantissa

For float this would imply a precision of 1/2^22 (or 0.00000011920928955078125) or a maximum
of 6 significant digits.  This backs up what I've seen experimentally.  I have also seen that
555.55499 (myFloat16) was rounded up by the precision specification "%.6g" to 555.555 and
not truncated to 555.554.

Bits Decimal
00   0.5
01   0.25
02   0.125
03   0.0625
04   0.03125
05   0.015625
06   0.0078125
07   0.00390625
08   0.001953125
09   0.0009765625
10   0.00048828125
11   0.000244140625
12   0.0001220703125
13   0.00006103515625
14   0.000030517578125
15   0.0000152587890625
16   0.00000762939453125
17   0.000003814697265625
18   0.0000019073486328125
19   0.00000095367431640625
20   0.000000476837158203125
21   0.0000002384185791015625
22   0.00000011920928955078125

My recommendation is that for serialisation of a float (this may also be necessary for double
and decimal), we use the following logic...

char * FloatToThreeDecimalPlaces( xsd__float myFloat)
{
    int                 sign = 1;

    if( myFloat < 0)
    {
        sign = -1;
        myFloat = -myFloat;
    }

    int                 whole = (int) myFloat;
    int                  fraction = (int) ((myFloat - (xsd__float) whole + (xsd__float) 0.00005)
* (xsd__float) 1000.0);
    static char   szNumber[16];

    sprintf( szNumber, "%d.%03d", whole * sign, fraction);

    char *    pZero = szNumber + strlen( szNumber) - 1;

    while( *pZero != '.' && *pZero == '0')
    {
        *pZero = '\0';
        pZero--;
    }

    if( *pZero == '.')
    {
        *pZero = '\0';
    }

    return szNumber;
}

Deserialisation
---------------------
We are using 'strtod' which appears to be working satisfactorily.  It should be noted though
that there will be some problems converting the output (which is double) to a float!


> The serialisation of 555.555 as a float produces a value of 555.554993 on the wire.
> -----------------------------------------------------------------------------------
>
>          Key: AXISCPP-838
>          URL: http://issues.apache.org/jira/browse/AXISCPP-838
>      Project: Axis-C++
>         Type: Bug
>   Components: Serialization
>  Environment: n/a
>     Reporter: Fred Preston
>     Assignee: Fred Preston

>
> Using the UnitTest_XSD_float test.  If you change the code as follows:-
> Was
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)35.353588);
> Now
> [40]// Test non-nillable element
> [41]	    xsd__float result = ws->asNonNillableElement((xsd__float)555.555);
> Then on the wire you get:-
> <nonNillableElement>555.554993</nonNillableElement>

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message