groovy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edmond Kemokai <ekemo...@gmail.com>
Subject Re: float/double calculation bug ?
Date Sun, 19 Mar 2017 16:13:56 GMT
Seems like floating point approximation at runtime. Interestingly enough
your code works with exact result when I try it but any minor reformulation
ends up with the approximate result. Also when I replace your computation
with literal integers I get the exact result...floating point arithmetic
can get messy so you'll have to think about what you're trying to
accomplish and perhaps consider rounding up/down as needed...I would avoid
words like mantissa or significand but basically you could choose a tiny
number and set it as the limit for rounding up in terms of the difference
between your floating point result and the next integral/integer value...I
don't want to get back into numerical analysis so I'll stop :)

I am the developer of HiveMind, it is a JSR223 based web app platform for
Groovy and other JVM languages. You can access:

url: http://demo.crudzilla.com:7000/
login: developer/developer

I have created a test file in: /com/crudzilla/cloudTest/web/txt8888/test.ste

You can test the code by right clicking on the test.ste file and selecting
"Open In Browser".

I have put your code in there, feel free to play with it and try out the
platform as a whole.

On Sun, Mar 19, 2017 at 11:28 AM, Derek Visch <derek.visch@gmail.com> wrote:

> Looks like floating point to me, what are you expecting?
>
> On Mar 19, 2017 10:04 AM, "Tx. T" <txt8888@yahoo.com> wrote:
>
>> Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
>>
>> testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle Corporation OS: Mac
OS X
>>
>> Mac OS X 10.12.3
>>
>>
>>
>> groovy:000>     def calc = { amount, ttl ->
>> groovy:001>         double rtn
>> groovy:002>         if (amount[-1] != '%') rtn = amount.toDouble()
>> groovy:003>         else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/, '').toFloat()
>> groovy:004>
>> groovy:004>         rtn
>> groovy:005>     }
>> ===> groovysh_evaluate$_run_closure1@39fcbef6*groovy:000> calc("70%", 330000)
>> ===> 230999.99999999997*
>> groovy:000> calc("10%", 330000)
>> ===> 33000.0
>> groovy:000> calc("20%", 330000)
>> ===> 66000.0
>> groovy:000> calc("30%", 330000)
>> ===> 99000.0
>> groovy:000> calc("40%", 330000)
>> ===> 132000.0
>> groovy:000> calc("50%", 330000)
>> ===> 165000.0
>> groovy:000> calc("60%", 330000)
>> ===> 198000.0*groovy:000> calc("70%", 330000)
>> ===> 230999.99999999997*
>> groovy:000> calc("80%", 330000)
>> ===> 264000.0
>> groovy:000> calc("90%", 330000)
>> ===> 297000.0
>> groovy:000> calc("100%", 330000)
>> ===> 330000.0
>>
>>

Mime
View raw message