nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DAVID SMITH <davidrsm...@btinternet.com.INVALID>
Subject Merging the unique attributes of 2 flowfiles
Date Mon, 01 Jun 2020 12:33:55 GMT
Hi
I have a group of log files coming in via http listener, up to 30 logs  per transaction,
of which I only need the values that are in 2 of those log files per transaction. After using
some RouteOnContents I end up with the two log flowfiles I want.
 In my current flow I am using a MergeContent processor to try and merge the two required
flowfiles on a common ident attribute value  which I have extracted from each log files earlier,
I have also extracted some other attributes from the flowfiles at this point, and as everything
I am interested in these attributes I don't mind what happens with the content of the flowfiiles.
When I step through the flow all is fine and works as I expect, however when I run it at pace
and log files are coming in for multiple transactions at the same time the merge fails on
most occasions. 

My mergecontent settings are:Merge Strategy                        Bin
Packing AlgorithmMerge Format                          Binary ConcatenationAttribute
Strategy                    Keep all Unique atttributesCorrelation Attribute
Name         ${import.ident}Metadata Strategy                   
Ignore MeatdataMinimum No Of Entries            2Maximum No Of Enteries           
2Max bin age                             1 minutes
All the other properties are at default.
Have I not set something correctly or is there a simpler way of merging the attributes from
two flowfiles onto one flowfile?
Many thanksDave

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message