nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <marka...@hotmail.com>
Subject Re: Potential 1.11.X showstopper
Date Thu, 06 Feb 2020 19:14:35 GMT
As Joe mentioned earlier in the thread, the way to track down a "too many open files" problem
is to run "lsof -p <pid>"
That will show all open files, and it often makes it pretty obvious what it is that's holding
the file handles open.

> On Feb 6, 2020, at 2:12 PM, Mike Thomsen <mikerthomsen@gmail.com> wrote:
> 
> Can you share a description of your flows in terms of average flowfile
> size, queue size, data velocity, etc.?
> 
> Thanks,
> 
> Mike
> 
> On Thu, Feb 6, 2020 at 1:59 PM Elli Schwarz
> <eliezer_schwarz@yahoo.com.invalid> wrote:
> 
>> We seem to be experiencing the same problems. We recently upgraded
>> several of our Nifis from 1.9.2 to 1.11.0, and now many of them are failing
>> with "too many open files". Nothing else changed other than the upgrade,
>> and our data volume is the same as before. The only solution we've been
>> able to come up with is to run a script to check for this condition and
>> restart the Nifi. Any other ideas?
>> Thank you!
>> 
>>    On Sunday, February 2, 2020, 9:11:34 AM EST, Mike Thomsen <
>> mikerthomsen@gmail.com> wrote:
>> 
>> Without further details, this is what I did to see if it was something
>> other than the usual issue of having not enough file handlers available.
>> Something like a legitimate case of someone forgetting to close file
>> objects or something in the code itself.
>> 
>> 1. Setup a 8core/32GB VM on AWS w/ Amazon AMI.
>> 2. Pushed 1.11.1RC1
>> 3. Pushed the RAM settings to 6/12GB
>> 4. Disabled flowfile archiving because I only allocated 8GB of storage.
>> 5. Setup a flow that used 2 generateflow instances to generate massive
>> amounts of garbage data using all available cores. (All queues were setup
>> to hold 250k flow files)
>> 6. Kicked it off and let it run for probably about 20 minutes.
>> 
>> No apparent problem with closing and releasing resources here.
>> 
>> On Sat, Feb 1, 2020 at 8:00 AM Joe Witt <joe.witt@gmail.com> wrote:
>> 
>>> these are usually very easy to find.
>>> 
>>> run lsof -p pid.  and share results
>>> 
>>> 
>>> thanks
>>> 
>>> On Sat, Feb 1, 2020 at 7:56 AM Mike Thomsen <mikerthomsen@gmail.com>
>>> wrote:
>>> 
>>>> 
>>>> 
>>> 
>> https://stackoverflow.com/questions/59991035/nifi-1-11-opening-more-than-50k-files/60017064#60017064
>>>> 
>>>> No idea if this is valid or not. I asked for clarification to see if
>>> there
>>>> might be a specific processor or something that is triggering this.
>>>> 
>>> 
>> 


Mime
View raw message