Return-Path: Delivered-To: apmail-ws-axis-c-dev-archive@www.apache.org Received: (qmail 35000 invoked from network); 23 Nov 2005 12:45:59 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 23 Nov 2005 12:45:59 -0000 Received: (qmail 2475 invoked by uid 500); 23 Nov 2005 12:45:59 -0000 Delivered-To: apmail-ws-axis-c-dev-archive@ws.apache.org Received: (qmail 2457 invoked by uid 500); 23 Nov 2005 12:45:58 -0000 Mailing-List: contact axis-c-dev-help@ws.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: List-Id: "Apache AXIS C Developers List" Reply-To: "Apache AXIS C Developers List" Delivered-To: mailing list axis-c-dev@ws.apache.org Received: (qmail 2442 invoked by uid 99); 23 Nov 2005 12:45:58 -0000 X-ASF-Spam-Status: No, hits=1.3 required=10.0 tests=SPF_FAIL X-Spam-Check-By: apache.org Received: from [192.87.106.226] (HELO ajax.apache.org) (192.87.106.226) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Nov 2005 04:45:58 -0800 Received: from ajax.apache.org (ajax.apache.org [127.0.0.1]) by ajax.apache.org (Postfix) with ESMTP id 3FC00239 for ; Wed, 23 Nov 2005 13:45:36 +0100 (CET) Message-ID: <1033839663.1132749936228.JavaMail.jira@ajax.apache.org> Date: Wed, 23 Nov 2005 13:45:36 +0100 (CET) From: "Fred Preston (JIRA)" To: axis-c-dev@ws.apache.org Subject: [jira] Commented: (AXISCPP-838) The serialisation of 555.555 as a float produces a value of 555.554993 on the wire. In-Reply-To: <1503753980.1127904707740.JavaMail.jira@ajax.apache.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/AXISCPP-838?page=comments#action_12358351 ] Fred Preston commented on AXISCPP-838: -------------------------------------- Precision problems with floats ------------------------------------------ Serialisation ----------------- Before trying to resolve anything, I though I would do some investigation into floats... I first created lots of float values around the 555.555 problem area to see how this number was being represented... Code "%f" Produced "%.7g" Produced "%.8g" Produced xsd__float myFloat00 = 555.5 + 0.0001; 555.500122 555.5001 555.50012 xsd__float myFloat01 = 555.55 + 0.0001; 555.550110 555.5501 555.55011 xsd__float myFloat02 = 555.555; 555.554993 555.555 555.55499 xsd__float myFloat020 = 555.555 + 0.1; 555.655029 555.655 555.65503 xsd__float myFloat021 = 555.555 + 0.01; 555.565002 555.565 555.565 xsd__float myFloat022 = 555.555 + 0.001; 555.556030 555.556 555.55603 xsd__float myFloat023 = 555.555 + 0.0001; 555.555115 555.5551 555.55511 xsd__float myFloat024 = 555.555 + 0.00001; 555.554993 555.555 555.55499 xsd__float myFloat03 = 555.5555 + 0.0001; 555.555603 555.5556 555.5556 xsd__float myFloat04 = 555.55555 + 0.0001; 555.555664 555.5557 555.55566 xsd__float myFloat05 = 555.556 + 0.0001; 555.556091 555.5561 555.55609 xsd__float myFloat06 = -555.5 - 0.0001; -555.500122 -555.5001 -555.50012 xsd__float myFloat07 = -555.55 - 0.0001; -555.550110 -555.5501 -555.55011 xsd__float myFloat08 = -555.555 - 0.0001; -555.555115 -555.5551 -555.55511 xsd__float myFloat09 = -555.5555 - 0.0001; -555.555603 -555.5556 -555.5556 xsd__float myFloat10 = -555.55555 - 0.0001; -555.555664 -555.5557 -555.55566 xsd__float myFloat11 = -555.556 - 0.0001; -555.556091 -555.5561 -555.55609 xsd__float myFloat12 = 555.5; 555.500000 555.5 555.5 xsd__float myFloat13 = 555.55; (5) 555.55 (6) 555.55 (7) 555.55 (8) 555.54999 xsd__float myFloat14 = 555.555; (6) 555.555 (7) 555.555 (8) 555.55499 xsd__float myFloat15 = 555.5555; (7) 555.5555 xsd__float myFloat16 = 555.55499; (6) 555.555 <- This has rounded up rather than truncated. (7) 555.555 <- This has rounded up rather than truncated. (8) 555.55499 xsd__float myFloat17 = 555.5499; (6) 555.55 <- This has rounded up rather than truncated. (7) 555.5499 (8) 555.54993 The best guaranteed precision without any rounding appears to be 5 digits and with rounding 6 (although I would need to confirm this). Reading through the IEEE numeric standards, types have the following precision:- float = sign bit, 8 bit exponent, 23 bit mantissa double = sign bit, 11 bit exponent, 52 bit mantissa long double = sign bit, 15 bit exponent, 64 bit mantissa For float this would imply a precision of 1/2^22 (or 0.00000011920928955078125) or a maximum of 6 significant digits. This backs up what I've seen experimentally. I have also seen that 555.55499 (myFloat16) was rounded up by the precision specification "%.6g" to 555.555 and not truncated to 555.554. Bits Decimal 00 0.5 01 0.25 02 0.125 03 0.0625 04 0.03125 05 0.015625 06 0.0078125 07 0.00390625 08 0.001953125 09 0.0009765625 10 0.00048828125 11 0.000244140625 12 0.0001220703125 13 0.00006103515625 14 0.000030517578125 15 0.0000152587890625 16 0.00000762939453125 17 0.000003814697265625 18 0.0000019073486328125 19 0.00000095367431640625 20 0.000000476837158203125 21 0.0000002384185791015625 22 0.00000011920928955078125 My recommendation is that for serialisation of a float (this may also be necessary for double and decimal), we use the following logic... char * FloatToThreeDecimalPlaces( xsd__float myFloat) { int sign = 1; if( myFloat < 0) { sign = -1; myFloat = -myFloat; } int whole = (int) myFloat; int fraction = (int) ((myFloat - (xsd__float) whole + (xsd__float) 0.00005) * (xsd__float) 1000.0); static char szNumber[16]; sprintf( szNumber, "%d.%03d", whole * sign, fraction); char * pZero = szNumber + strlen( szNumber) - 1; while( *pZero != '.' && *pZero == '0') { *pZero = '\0'; pZero--; } if( *pZero == '.') { *pZero = '\0'; } return szNumber; } Deserialisation --------------------- We are using 'strtod' which appears to be working satisfactorily. It should be noted though that there will be some problems converting the output (which is double) to a float! > The serialisation of 555.555 as a float produces a value of 555.554993 on the wire. > ----------------------------------------------------------------------------------- > > Key: AXISCPP-838 > URL: http://issues.apache.org/jira/browse/AXISCPP-838 > Project: Axis-C++ > Type: Bug > Components: Serialization > Environment: n/a > Reporter: Fred Preston > Assignee: Fred Preston > > Using the UnitTest_XSD_float test. If you change the code as follows:- > Was > [40]// Test non-nillable element > [41] xsd__float result = ws->asNonNillableElement((xsd__float)35.353588); > Now > [40]// Test non-nillable element > [41] xsd__float result = ws->asNonNillableElement((xsd__float)555.555); > Then on the wire you get:- > 555.554993 -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira