nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: Reg: Issues in Custom Processor
Date Wed, 27 Apr 2016 14:27:34 GMT
Hi Sourav,

Are you able to share any of the code from your onTrigger method so we can
see how you are opening/closing the file reader?

-Bryan

On Wed, Apr 27, 2016 at 7:43 AM, Sourav Gulati <sourav.gulati@impetus.co.in>
wrote:

> I have introduced yield of 5 secs and after that it started working as
> expected.
>
> Regards,
> Sourav Gulati
>
> -----Original Message-----
> From: Sourav Gulati [mailto:sourav.gulati@impetus.co.in]
> Sent: Wednesday, April 27, 2016 11:34 AM
> To: dev@nifi.apache.org
> Subject: RE: Reg: Issues in Custom Processor
>
> Joe,
>
> Now I have increased number of open files limit.
> ulimit -n
> 50000
>
> How still I am seeing same exception
>
> Regards,
> Sourav Gulati
>
> -----Original Message-----
> From: Sourav Gulati [mailto:sourav.gulati@impetus.co.in]
> Sent: Tuesday, April 26, 2016 11:48 PM
> To: dev@nifi.apache.org
> Subject: Re: Reg: Issues in Custom Processor
>
> Thanks Joe. I will try and update you
>
> Regards,
> Sourav Gulati
> Big Data
>
>
> ________________________________________
> From: Joe Witt <joe.witt@gmail.com>
> Sent: Tuesday, April 26, 2016 11:26 PM
> To: dev@nifi.apache.org
> Subject: Re: Reg: Issues in Custom Processor
>
> Sourav,
>
> On twitter you asked "why is it opening so many file handles"
>
> Your processor is not.  It may well be just fine. However, at startup nifi
> loads many files into the classloader and it is at the point now where it
> references more than 1024 objects (files, sockets, etc..) so it is
> important to set a higher default.
>
> I encourage you to go ahead and follow the best practices mentioned in the
> documentation and I suspect you'll find your custom processor is doing what
> you expected.
>
> Thanks
> Joe
>
> On Tue, Apr 26, 2016 at 1:47 PM, Joe Witt <joe.witt@gmail.com> wrote:
> > Sourav,
> >
> > Please make sure you've followed these recommendations
> > https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#
> > configuration-best-practices
> >
> > Specifically make sure you update the max open file handles.
> >
> > Thanks
> > Joe
> >
> > On Tue, Apr 26, 2016 at 9:34 AM, Sourav Gulati
> > <sourav.gulati@impetus.co.in> wrote:
> >> Hi Team,
> >>
> >> I am using "@TriggerSerially"  annotation while creating processor and
> inside "On Trigger"  method I am creating a file reader and after doing the
> processing I am closing the reader connection as well. As per my
> assumption, if I am using @TriggerSerially, then "On trigger" method runs
> in serial fashion.
> >>
> >> However, while executing processor I am getting "too many open files"
> error . I check with lsof command as well, it has opened lot of reader
> connections to file.
> >>
> >> What should I change ? Need urgent help
> >>
> >>
> >>
> >> Regards,
> >> Sourav Gulati
> >>
> >>
> >> ________________________________
> >>
> >>
> >>
> >>
> >>
> >>
> >> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message