nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Gibeault (pagibeault)" <pagibea...@micron.com>
Subject RE: attributesToJSON order
Date Fri, 21 Oct 2016 21:13:28 GMT
Bryan and Aldrin,
  Thanks for your replies.

The user has already specified an order in the attributesToJSON processor’s configuration.
 If we honor this order, we should be covered.  Changing the datatype to LinkedHashMap is
exactly what I had in mind.  I do not believe we need to worry about impacting legacy behavior
here as users are already accustomed to not depending on the order of the JSON values.

This became an issue for us when, downstream, we used a custom processor JSONToDelimited to
export the JSON content as a CSV string.  We were unable to trust the order of the JSON elements
which resulted in invalid ordering of the values in our CSV.

We have plans to provide additional configuration on our JSONToDelimited processor to specify
which JSON keys to export and in which order.  In the meantime, we are using the replaceText
processor with the attributes called out explicitly.  This is messy but effective.

Thanks,
Paul Gibeault

From: Bryan Rosander [mailto:bryanrosander@gmail.com]
Sent: Thursday, October 20, 2016 1:43 PM
To: users@nifi.apache.org
Cc: Peter Wicks (pwicks) <pwicks@micron.com>
Subject: Re: attributesToJSON order

I'm not sure of the use case either but if there is one, it should be pretty easy to let the
user select the type of Map they'd prefer to store it in.

HashMap should probably be default if order doesn't matter, LinkedHashMap if you want to maintain
insertion order, TreeMap if you want alphabetical order.

Thanks,
Bryan

On Thu, Oct 20, 2016 at 3:27 PM, Aldrin Piri <aldrinpiri@gmail.com<mailto:aldrinpiri@gmail.com>>
wrote:
Hey Paul,

Could you highlight the use case you are looking to address or shortcoming that has emerged
because of this?  No strong qualms with providing it, just not sure I am tracking where this
becomes problematic.

Thanks,
Aldrin

On Thu, Oct 20, 2016 at 3:23 PM, Paul Gibeault (pagibeault) <pagibeault@micron.com<mailto:pagibeault@micron.com>>
wrote:
Hello all,
  I thought I would run this by you before I created a Jira ticket.

The processor attributesToJSON does not create a JSON document with key/values in the same
order as provided in the processor’s configuration.

Example:
  AttributesList: computationName,computationType,strategyName

Output:
{
  " strategyName" : "blue",
  " computationType" : "21DC8X32",
  " computationName" : "453d6c4f-fdd-e611-80c9-0050233e88"
}

This behavior is coming from the datatype used in the attributesToJSON processor:

    protected Map<String, String> buildAttributesMapForFlowFile(FlowFile ff, String
atrList,
                                                                boolean includeCoreAttributes,
                                                                boolean nullValForEmptyString)
{

        Map<String, String> atsToWrite = new HashMap<>();

        . . .
    }

Using another datatype that preserved order would correct this behavior.  The JSON specification
does mention that the object list is order independent.  This does not necessarily mean we
should cause the disorder though.

Should we create a JIRA ticket and solution for this?

Thanks,
Paul Gibeault



Mime
View raw message