nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sourav Gulati <sourav.gul...@impetus.co.in>
Subject RE: Reg: Issues in Custom Processor
Date Wed, 27 Apr 2016 11:43:34 GMT
I have introduced yield of 5 secs and after that it started working as expected.

Regards,
Sourav Gulati

-----Original Message-----
From: Sourav Gulati [mailto:sourav.gulati@impetus.co.in]
Sent: Wednesday, April 27, 2016 11:34 AM
To: dev@nifi.apache.org
Subject: RE: Reg: Issues in Custom Processor

Joe,

Now I have increased number of open files limit.
ulimit -n
50000

How still I am seeing same exception

Regards,
Sourav Gulati

-----Original Message-----
From: Sourav Gulati [mailto:sourav.gulati@impetus.co.in]
Sent: Tuesday, April 26, 2016 11:48 PM
To: dev@nifi.apache.org
Subject: Re: Reg: Issues in Custom Processor

Thanks Joe. I will try and update you

Regards,
Sourav Gulati
Big Data


________________________________________
From: Joe Witt <joe.witt@gmail.com>
Sent: Tuesday, April 26, 2016 11:26 PM
To: dev@nifi.apache.org
Subject: Re: Reg: Issues in Custom Processor

Sourav,

On twitter you asked "why is it opening so many file handles"

Your processor is not.  It may well be just fine. However, at startup nifi loads many files
into the classloader and it is at the point now where it references more than 1024 objects
(files, sockets, etc..) so it is important to set a higher default.

I encourage you to go ahead and follow the best practices mentioned in the documentation and
I suspect you'll find your custom processor is doing what you expected.

Thanks
Joe

On Tue, Apr 26, 2016 at 1:47 PM, Joe Witt <joe.witt@gmail.com> wrote:
> Sourav,
>
> Please make sure you've followed these recommendations
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#
> configuration-best-practices
>
> Specifically make sure you update the max open file handles.
>
> Thanks
> Joe
>
> On Tue, Apr 26, 2016 at 9:34 AM, Sourav Gulati
> <sourav.gulati@impetus.co.in> wrote:
>> Hi Team,
>>
>> I am using "@TriggerSerially"  annotation while creating processor and inside "On
Trigger"  method I am creating a file reader and after doing the processing I am closing the
reader connection as well. As per my assumption, if I am using @TriggerSerially, then "On
trigger" method runs in serial fashion.
>>
>> However, while executing processor I am getting "too many open files" error . I check
with lsof command as well, it has opened lot of reader connections to file.
>>
>> What should I change ? Need urgent help
>>
>>
>>
>> Regards,
>> Sourav Gulati
>>
>>
>> ________________________________
>>
>>
>>
>>
>>
>>
>> NOTE: This message may contain information that is confidential, proprietary, privileged
or otherwise protected by law. The message is intended solely for the named addressee. If
received in error, please destroy and notify the sender. Any use of this email is prohibited
when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity
of this communication has been maintained nor that the communication is free of errors, virus,
interception or interference.

________________________________






NOTE: This message may contain information that is confidential, proprietary, privileged or
otherwise protected by law. The message is intended solely for the named addressee. If received
in error, please destroy and notify the sender. Any use of this email is prohibited when received
in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors, virus, interception
or interference.

________________________________






NOTE: This message may contain information that is confidential, proprietary, privileged or
otherwise protected by law. The message is intended solely for the named addressee. If received
in error, please destroy and notify the sender. Any use of this email is prohibited when received
in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors, virus, interception
or interference.

________________________________






NOTE: This message may contain information that is confidential, proprietary, privileged or
otherwise protected by law. The message is intended solely for the named addressee. If received
in error, please destroy and notify the sender. Any use of this email is prohibited when received
in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors, virus, interception
or interference.

Mime
View raw message