nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: StateManager race condition potential
Date Fri, 09 Aug 2019 18:11:33 GMT
Russell

As I understand it the state is specific to an instance of a processor on
the flow.  Your safety ends there.  If you allow that processor to have
multiple tasks running concurrently then you'll need to protect usage of
that state mechanism just as you would with any other variable in the scope
of the processor.  If the scope is local then the above is really all you
need to think about.  If the scope is cluster wide then generally speaking
I think the intent of usage for that often is associated with a primary
node only thing like a ListX processor (ListFile) with the idea being that
the state can be restored by some other node if that could get assigned
that role.  I'm not clear on the role of cluster wide state otherwise.
Others will have to comment on that.

Thanks

On Fri, Aug 9, 2019 at 1:54 PM Russell Bateman <russ@windofkeltia.com>
wrote:

> I'm assuming that the StateManagerprotects itself against race
> conditions for the consuming (custom) processor, but I'd like
> confirmation on that. Let's say something simple like we get an integer
> out of state to which we can add one to get the next (piece of work to
> do), then immediately bump and write that value plus 1 for the next
> thread to get. In the time it took us to get the value back, bump it by
> 1, then put it out (I'm assuming Scope.LOCAL), I don't see that the
> StateManageris prevented from handing out that same value to another
> instance or task of my processor.
>
> How does StateManager Scopeaffect this? (By whether the instance of
> state is per host or per cluster?)
> How does processor behavior annotation affect this?
> How does processor scheduling configuration (concurrent task count)
> affect this?
>
> Thanks for any comments.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message