nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Payne (JIRA)" <>
Subject [jira] [Commented] (NIFI-190) HoldFile processor
Date Mon, 23 Mar 2015 12:58:11 GMT


Mark Payne commented on NIFI-190:


This would be possible, but I'm not sure that it's really the right abstraction. The DistributedMapCacheClient
was designed to communicate over the the network so that all nodes in a cluster have the same
view of the map. It also requires that the NCM (or some node) be running a DistributedMapCacheServer.
So it's a little bit overkill and would be expensive for each node to continually communicate
with the NCM when we expect the FlowFile to be local.

Or am I missing something?

> HoldFile processor
> ------------------
>                 Key: NIFI-190
>                 URL:
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: Extensions
>            Reporter: Joseph Gresock
>            Priority: Minor
>         Attachments: HoldFile_example.xml
> Our team has developed a processor for the following use case:
> * Format A needs to be sent to Endpoint A
> * Format B needs to be sent to Endpoint B, but should not proceed until A has reached
Endpoint A.  We most commonly have this restriction when Endpoint B requires some output of
Endpoint A.
> The proposed HoldFile processor takes 2 types of flow files as input:
> * Files to be held
> * Signal files that can release corresponding held files, based on the value of a configurable
"release" attribute
> Signal files are distinguished from held files by the presence of the "flow.file.release.value"
attribute.  The processor is configured with a "Release Signal Attribute".  Held files with
this attribute whose value matches a received signal value will be released.
> An example:
> HoldFile is configured with Release Signal Attribute = "myId".  Its 'Hold' relationship
routes back onto itself.
> 1. flowFile 1 { myId : "123" } enters HoldFile.  It is routed to the 'Hold' relationship
> 2. flowFile 2 { flow.file.release.value : "123" } enters HoldFile.  flowfile 1 is then
routed to 'Release', and flow file 2 is removed from the session.
> Signal flow files will also copy their attributes to matching held files, unless otherwise
indicated.  This is what allows the output of Endpoint A to pass to Endpoint B, above.

This message was sent by Atlassian JIRA

View raw message