nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe Witt (Jira)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-7114) NiFi not closing file handles
Date Fri, 21 Feb 2020 14:39:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-7114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041924#comment-17041924
] 

Joe Witt commented on NIFI-7114:
--------------------------------

So far I do not see a true leak.  However the output difference between lsof and lsof -p is
pretty interesting.

lsof without specifying pid but using -E or +E gives more details.  And for every thread of
nifi there appears to be a correlation between the 5800 open files you'd see from lsof -p
to the 580000 open files you'd see from lsof without the pid.  The pattern repeats for every
thread showing output like 

5735 228207 228339 REG
   7 228207 228339 unix
   16 228207 228344 a_inode
   7 228207 228344 CHR
   4 228207 228344 DIR
   33 228207 228344 FIFO
   10 228207 228344 IPv4
  5735 228207 228344 REG
   7 228207 228344 unix
   16 228207 228606 a_inode
   7 228207 228606 CHR
   4 228207 228606 DIR
   33 228207 228606 FIFO
   10 228207 228606 IPv4

In that output 228207 is the pid of nifi.  still hunting but dont quite get it

> NiFi not closing file handles
> -----------------------------
>
>                 Key: NIFI-7114
>                 URL: https://issues.apache.org/jira/browse/NIFI-7114
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Configuration
>    Affects Versions: 1.10.0, 1.11.0
>         Environment: Amazon EC2 running either Amazon Linux 2 or Ubuntu 18.04.
> NiFi has been installed with no change to any configuration file.
>            Reporter: Vinicius Zolin
>            Priority: Major
>         Attachments: destination.xml, fifocounts.txt, flow.xml.gz, lsof.log, lsof.zip,
lsofAfter.log, lsofBefore.log, openFiles.xlsx, reproduction.zip, source.xml
>
>
> Since at least version 1.10 NiFi stopped closing file handles. It opens circa 500 files
per hour (measured using lsof) without any apparent limit until it crashes due to too many
open files.
>  
> Increasing the computer open file limit is not a solution since NiFi will still crash,
it'll only take longer to do so.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message