nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Grande <agra...@hortonworks.com>
Subject Re: Refer to original FlowFile after AttributesToJSON processor
Date Thu, 28 Apr 2016 15:28:38 GMT
Igor, this is a common pattern for such systems, you need to 'save' old content in advance.
E.g. as a property of the message, then you can transform the content and still have access
to the original value (via an attribute value on the message).

Keep in mind, though, that you don't want to put large values into attributes. Alternatively
you would have to design you solution so that you could store the content somewhere explicitly
and look it up (e.g. have my_content_old & my_content_new as lookup keys). NiFi's DistributedCache
facility might be a good fit for such data.

Andrew

From: Igor Kravzov <igork.inexso@gmail.com<mailto:igork.inexso@gmail.com>>
Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Date: Monday, April 25, 2016 at 9:58 PM
To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Subject: Refer to original FlowFile after AttributesToJSON processor

Is there a way to refer to original FlowFile after AttributesToJSON  call? Destination set
to flowfile-content.
Or is there a way to produce a FlowFile from an attribute?

I need to extract some properties from Twitter JSON, convert to a new JSON, and after to extract
another property from original JSON.

I can't do it before because this property will be included in a newly generated JSON.

I am still battling with extracting "entities" entry from Twitter JSON. I also extract "text"
from tweet and found out if I just blindly replace escaped JSON after conversion it brakes
JSON in case when "text" contains quotes.

So my idea is to put some kind of placeholder parameter and replace it after conversion with
original "entity" value.

Hope I explained problem good enough.

Thanks in dvance.
Mime
View raw message