nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Wing <jvw...@gmail.com>
Subject Re: Combining outputs in parallel returns a random output
Date Thu, 02 Jun 2016 06:03:54 GMT
Kavinaesh,

I believe your "parallel" flow actually generates three separate flow files
from the three match outputs of EvaluateJSONPath "Get File Name alone".  If
you stop the AttributesToJSON processor "Create Json" and examine the
contents of the three input queues, I believe you will find they are
independent FlowFiles with separate UUID attributes, although they will
share the same "filename" attribute and underlying content (until
modified).  But their attributes are unique.  Your sequential flow works
correctly because the attributes are cumulatively modified on the same one
flow file.

You might be able to fix the parallel flow by using a MergeContent
processor immediately before the AttributesToJSON "Create Json" with
settings like the following:

   - *Attribute Strategy*: Keep All Unique Attributes
   - *Correlation Attribute Name*: filename
   - *Minimum Number of Entries*: 3


That will merge 3 flow files that share the same "filename" attribute
value, with a superset of their attributes.  Is that what you are looking
for?

Thanks,

James

On Wed, Jun 1, 2016 at 10:25 PM, Kavinaesh Dheenadayalan <
kavinaesh@gmail.com> wrote:

> Hi Team,
>
> I am new to nifi and working on a demo to my colleagues.
>
> I am creating a json by using this flow. The outputs of
> ExecuteStreamCommand processor which are json attributes are combined to
> create a json.
> When these three ExecuteStreamCommand processors are connected
> sequentially to an AttributesToJSON processor, the flow works fine with all
> the three attributes getting populated in the final json.
> But when I connect these three ExecuteStreamCommand processors in
> parallel, only one random attribute gets populated in the final json.
> Please let me know if there is a way to connect these processors in
> parallel and get all the attributes in the final json. scheduling or
> merging or any other option?
>
> I have attached the templates, screen shots of the processors for your
> reference.
>
> *Wrong output when the flow is parallel:*
>
> { "checks": [  {  "Check_Type": "File_Delimiter_Validation",  *"value":
> null* }, { "Check_Type": "File_Header_Validation", "value": "0" },  {
> "Check_Type": "File_Trailer_Validation",  *"value": null *} ],
> "fileName": "input.txt" }
>
>
> *Correct output when the flow is sequential:*
>
> { "checks": [  {  "Check_Type": "File_Delimiter_Validation",  *"value":
> "50"* }, { "Check_Type": "File_Header_Validation", "value": "0" },  {
> "Check_Type": "File_Trailer_Validation",  *"value": "100" *} ],
> "fileName": "input.txt" }
>
>

Mime
View raw message