nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Kelly (Jira)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-7114) NiFi not closing file handles
Date Fri, 21 Feb 2020 13:33:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-7114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041864#comment-17041864
] 

Paul Kelly commented on NIFI-7114:
----------------------------------

That is so strange that you can't replicate.  The simpler flow leaks 100% of the time for
me when stopping and starting GenerateFlowFile for one minute each.

There is more to the story with our production flow.  This test flow is not what held me
off of upgrading as I did upgrade in production.  We leak so badly there that I originally
set up a script to restart NiFi once per hour since by two hours we'd hit our increased ulimit
of 100,000 and NiFi would hang with "out of file handles".

Maybe it's something related to the version of Linux we're using?  I'm using Ubuntu Server
18.04 and Xubuntu 18.04 for testing, which is what [~vzolin] reported using as well.

> NiFi not closing file handles
> -----------------------------
>
>                 Key: NIFI-7114
>                 URL: https://issues.apache.org/jira/browse/NIFI-7114
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Configuration
>    Affects Versions: 1.10.0, 1.11.0
>         Environment: Amazon EC2 running either Amazon Linux 2 or Ubuntu 18.04.
> NiFi has been installed with no change to any configuration file.
>            Reporter: Vinicius Zolin
>            Priority: Major
>         Attachments: destination.xml, fifocounts.txt, flow.xml.gz, lsof.log, lsof.zip,
lsofAfter.log, lsofBefore.log, openFiles.xlsx, reproduction.zip, source.xml
>
>
> Since at least version 1.10 NiFi stopped closing file handles. It opens circa 500 files
per hour (measured using lsof) without any apparent limit until it crashes due to too many
open files.
>  
> Increasing the computer open file limit is not a solution since NiFi will still crash,
it'll only take longer to do so.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message