nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From l...@china-inv.cn
Subject 答复: Re: 答复: Re: NiFi starts very slowly
Date Fri, 29 Dec 2017 01:54:09 GMT
Thanks for your help.

Actually we are using Kylo (https://kylo.io/) which uses NiFi to do the 
work.

There are about 10 processors in each group, so there will be 16000 
processors in total.

I understand that the processor should be validate before it starts, but 
could this validation be defered until the processor (or the group it 
belongs to) is to start for the first time
instead of doing this when NiFi service starts up?



发件人: 
Joe Witt <joe.witt@gmail.com>
收件人:
dev@nifi.apache.org
日期:
2017/12/29 09:20
主题:
Re: 答复: Re: NiFi starts very slowly



hello

the total number of processors is probably a factor.  how many are there?
can you please list how many processors of each type there are?  Any 
custom
processors?

creating 1600 disjoint flows is probably going against the grain of nifi.
cant the flows be designed to operate as a directed graph amd leverage
attributes?

keep in mind to start each processor we call validate.  this could be the
cause of the slowness.

help us collect the above info and we can give pointers.  if you could
share your flow.xml.gz that would be great.

thanks
joe

On Dec 28, 2017 8:13 PM, <luby@china-inv.cn> wrote:

> Thanks for your reply.
>
> We are not using a cluster, our NiFi 1.3 runs in the single node mode.
>
>
>
> 发件人:
> "Mohammed Nadeem" <nadeemm410@gmail.com>
> 收件人:
> dev@nifi.apache.org
> 日期:
> 2017/12/28 20:11
> 主题:
> Re: NiFi starts very slowly
>
>
>
> Hi Luby,
>
> If you are using a cluster based setup then Nifi is unstable due to 
nodes
> getting disconnected by the cluster cordinator. Cluster coordinator
> disconnects nodes from the cluster when the nodes take a little longer 
to
> respond ( beyond timeout) . The cluster coordinator timeout based on the
> settings in the nifi.properties file.
>
> We need to adjust the following properties
>
> *nifi.cluster.node.connection.timeout * = 5 sec  ( by default )
> *nifi.cluster.node.read.timeout *=  5 sec ( by default)
>
> By Increasing the timeout ,from 5 sec to higher value would probably 
solve
> a
> problem
>
>
> Thanks,
> Nadeem
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>
>
>
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人,请您不要阅读、保存、对
外
> 披露或复制本邮件的任何内容,或者打开本邮件的任何附件。请即回复邮件告知发
件
> 人,并立刻将该邮件及其附件从您的电脑系统中全部删除,不胜感激。
>
>
> This email message may contain confidential and/or privileged 
information.
> If you are not the intended recipient, please do not read, save, 
forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender 
immediately
> by replying this email, and delete this email and any attachment or 
links
> to this email completely and immediately from your computer system.
>
>
>
>





 
本邮件内容包含保密信息。如阁下并非拟发送的收件人,请您不要阅读、保存、对外
披露或复制本邮件的任何内容,或者打开本邮件的任何附件。请即回复邮件告知发件
人,并立刻将该邮件及其附件从您的电脑系统中全部删除,不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message