nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <marka...@hotmail.com>
Subject Re: Slow flowfile transfer from process group port to output port.
Date Tue, 25 Sep 2018 15:34:03 GMT
Ashwin,

You'll want to Right-Click on the RPG and then choose to configure Remote Ports. From there,
you can
configure how many threads should be used to pull data from each port. So you've updated the
Output Port
to use up to 3 threads per node to provide the data, but each node is still only using 1 thread
to pull the data.
By configuring the Remote Ports, you can configure the RPG to use 3 threads to pull the data
as well.

Thanks
-Mark


> On Sep 25, 2018, at 11:29 AM, ashwin konale <ashwin.konale@gmail.com> wrote:
> 
> Hi I have the following flow which I am trying to increase the throughput.
> I am runnign 3 node nifi cluster(v1.6)
> 
> CDC -> processGroupOutputPort -> externalOutputPort
> RPG -> downstream
> 
> Flowfiles are always queued up between processGroupOutputPort to
> externalOutputPort. I dont see any configuration to increase the number of
> threads for this operation as well. I have set backpressure to be 10,000
> flowfiles. Since the CDC produces at much faster rate than this, I want to
> increase the throughput of the flow. Also I see in the UI
> externalOutputPort has number of threads running as 3 all the time no
> matter how much I increase number of threads to consume in RPG.
> 
> Any pointers here will be much helpful.
> 
> Ashwin


Mime
View raw message