nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Janssen <bjanss...@gmail.com>
Subject Asynchronous Request/Response Data Flows
Date Thu, 26 Jan 2017 20:10:53 GMT
Hello all,

I've got a use case where I get some data, I want to fork a portion of that
data off to an external service for asynchronous processing, and when that
external service has finished processing the data, I want to take its
output, marry it up with the original data, and pass the whole thing on for
further processing.

So essentially two data flows:

Receive Data -> Store Some State -> Send Data To External Service

Do More Processing On Original Data + Results <-  Retrieve Previously
Stored State  <-  Receive Results From External Service

Is there a way to do this while taking advantage of NiFi's State Management
capabilities?  I wasn't finding any obvious processors for persisting and
retrieving shared state from State Management.  The closest my googling was
able to get me was this:  https://issues.apache.org/jira/browse/NIFI-1582
but if I'm understanding the State Management documentation properly, that
won't actually help me because I'd need the same processor to do all
storing and retrieving of state?

Does something exist to use State Management like this?  Or is what I'm
proposing to do a bad idea?

Or maybe I should just be using the DistributedMapCacheServer for this?

Any help/advice would be appreciated.

Mime
View raw message