nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milan Das <m...@interset.com>
Subject Re: Controlling number of threads in distributed mode
Date Thu, 14 Feb 2019 02:52:02 GMT
Hi Joe,
Use case is I have to query HTTP request.  The HTTP  request  from  NIFI can only run 3 queries
at a time.

The queries to be executed are coming from Kafka topic and we have 6 node cluster.
Ideally would like to run the 3 http queries from 3 different nodes otherwise one of the node
(primary node) will be overloaded with data.
Thanks,
Milan Das



> On Feb 13, 2019, at 4:38 PM, Joe Witt <joe.witt@gmail.com> wrote:
> 
> Milan
> 
> The only mechanism to limit whether a given component executes on a given
> nifi node or not is whether it runs on all or whether it runs on primary
> node.
> 
> Can you describe the use case that would lead to wanting to run a component
> on a specified number of nodes as opposed to all nodes?  It would defy the
> clustering logic/data distribution mechanisms of site to site and load
> balanced connections for example.
> 
> Thanks
> 
> On Wed, Feb 13, 2019 at 4:34 PM Milan Das <mdas@interset.com> wrote:
> 
>> Hello,
>> Is there a way to control number of thread less than cluster member ?
>> I have a 5 node NiFI cluster and I want to run 3 instances of the
>> processor in distributed mode.
>> 
>> Thanks,
>> Milan Das
>> 
>> 


Mime
View raw message