nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy LoPresto <alopre...@apache.org>
Subject Re: [Discuss] Expression Language Options to Add Decimal Support
Date Thu, 21 Apr 2016 02:47:17 GMT
I would support option 4, then option 1. I think this is a significant change, so backward-compatibility
is important, and I don’t see specifying the return type to be a high hurdle provided the
documentation is clear. If someone is writing in expression language, there is a minimum expectation
of exposure to the documentation, as it is a bespoke implementation, not an OTS offering that
they would have previous experience with.

I think options 2 and 3 introduce the opportunity to compromise previously-functioning expressions
in a way that could be very difficult to troubleshoot.

I do appreciate you discovering and diagnosing this. I can imagine there are already scenarios
where EL is failing silently and this should definitely be fixed correctly.


Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Apr 20, 2016, at 7:16 PM, Joe Percivall <joepercivall@yahoo.com.INVALID> wrote:
> 
> Hello Dev list,
> I've been working on a couple improvements that deal with utilizing expression language
to do data analysis and this has exposed a couple issues with the typing of numbers in Expression
Language (EL). The improvements are a continuation of the topic I presented on at the Apache
NiFi MeetUp group in MD[1].
> The primary issue I came across is that currently in EL all numbers interpreted as longs[2],
which only store whole numbers. This leads to problems when trying to do things like dividing
whole numbers or just trying to add/subtract decimals. I am actually surprised that the request
for decimals this hasn't come up before. That being said, after some initial discussion with
Tony and Joe, I believe that there are four potential ways forward.
> 1: Create a new EL type "decimal" backed by a double[3] and new methods to support it
("add_decimal"): This allows the user to explicitly choose whether or not they want to use
decimal or whole numbers. It retains the simple use-cases that use whole numbers while opening
up the new use-cases using decimals. One down side is that it is more verbose. It means adding
a new function for each math operation. This is backwards compatible.
> 
> 2: Back all numbers by doubles instead of longs: The easy to implement and retains very
concise methods ("add", "subtract", etc..). A few cons, doubles have a lower precision than
longs[4],  can lead to rounding errors[5] and could be confusing for users who only work with
whole numbers to suddenly find decimals.This is not backwards compatible.
> 3: Create a new EL type "decimal" back by a double[3] and attempt to smartly automatically
cast depending on method/input/output: This would allow for the positives of having decimals
and whole numbers in addition to having concise methods. The main cons being a much longer
implementation time to do it right and the "shadiness" of doing things automatically for the
user. Also this would mean the user wouldn't have the option to explicitly override  This
is not backwards compatible.
> 4: Create a new EL type "decimal" backed by a double[3] and overload the existing methods
with an additional parameter to specify return type to support it: This would allow for the
positives of having decimals and whole numbers in addition to having concise method names
but this may cause confusion with less technical users who aren't used to specifying return
types. This is backwards compatible.
> 
> The options that are not backwards compatible would need to wait to be implemented until
1.0.
> The current option I am leaning towards is number 1 due to the explicitness and greater
control it gives the user. While it is more verbose I think the decimal vs whole number syntax
will be easy for even non-technical users to pick up. Also I currently have a PR for it up
here[6].
> Any other ideas or suggestions are welcome!
> [1] http://www.meetup.com/ApacheNiFi/events/229158777/
> [2] https://docs.oracle.com/javase/7/docs/api/java/lang/Long.html
> [3] https://docs.oracle.com/javase/7/docs/api/java/lang/Double.html
> [4] https://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html
> [5] http://stackoverflow.com/questions/960072/rounding-errors
> [6] https://issues.apache.org/jira/browse/NIFI-1662
> Joe- - - - - - Joseph Percivalllinkedin.com/in/Percivalle: joepercivall@yahoo.com


Mime
View raw message