nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe Witt (Jira)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-7114) NiFi not closing file handles
Date Fri, 21 Feb 2020 15:20:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-7114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041946#comment-17041946
] 

Joe Witt commented on NIFI-7114:
--------------------------------

[~pkelly.nifi] I dont believe the lsof command you're running is giving you accurate data.
 lsof without the pid is showing the same things over and over because it is doing it for
every single thread as a reflection of the original process.  I have confirmed this relationship
by adding threads to the controller service which reproduces the 'leak' but that is only a
leak in the sense that lsof without the p will show every open file handle the process actually
has for every thread.  Since you're filtering on FIFO you're seeing that value grow which
is a reflection of each thread.

Please use lsof -p <pid> and let us know if you're seeing that grow over time and if
so please share the output of that command.  Ideally it would be 

lsof +E -P -p <pid of nifi not bootstrap>



> NiFi not closing file handles
> -----------------------------
>
>                 Key: NIFI-7114
>                 URL: https://issues.apache.org/jira/browse/NIFI-7114
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Configuration
>    Affects Versions: 1.10.0, 1.11.0
>         Environment: Amazon EC2 running either Amazon Linux 2 or Ubuntu 18.04.
> NiFi has been installed with no change to any configuration file.
>            Reporter: Vinicius Zolin
>            Priority: Major
>         Attachments: destination.xml, fifocounts.txt, flow.xml.gz, lsof.log, lsof.zip,
lsofAfter.log, lsofBefore.log, openFiles.xlsx, reproduction.zip, source.xml
>
>
> Since at least version 1.10 NiFi stopped closing file handles. It opens circa 500 files
per hour (measured using lsof) without any apparent limit until it crashes due to too many
open files.
>  
> Increasing the computer open file limit is not a solution since NiFi will still crash,
it'll only take longer to do so.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message