nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Bence (Jira)" <>
Subject [jira] [Assigned] (NIFI-7454) Directory handle leak in ListFile
Date Fri, 15 May 2020 11:38:00 GMT


Simon Bence reassigned NIFI-7454:

    Assignee: Simon Bence

> Directory handle leak in ListFile
> ---------------------------------
>                 Key: NIFI-7454
>                 URL:
>             Project: Apache NiFi
>          Issue Type: Bug
>    Affects Versions: 1.11.4
>            Reporter: Paul Kelly
>            Assignee: Simon Bence
>            Priority: Major
>         Attachments: flow.xml.gz
> ListFile does not seem to be closing directory handles if an exception is raised during
the directory walk.  For example, if ListFile is recursively polling a folder and the NiFi
user does not have permission to list the contents of one of the subfolders, an "Access Denied"
bulletin is raised and the number of DIR handles opened by the NiFi process as shown by lsof
increases.  Eventually, NiFi stops working due to "Too many open files".
> This has been tested with NiFi 1.11.4 on Ubuntu 18.04 with Java 11.  I'm not sure in
which version this behavior started.
> Attached is a sample flow.xml.gz that demonstrates this behavior.  A clean copy of NiFi
1.11.4 was installed with no modifications to any configuration file.  NiFi is running as
the user "xubuntu".  The permissions on the folders are as follows:
> drwxrwxr-x 3 xubuntu xubuntu 4096 May 15 08:55 /home/xubuntu/test
> drwxrwx--- 2 root root 4096 May 15 08:55 /home/xubuntu/test/subtest
> NiFi has permission to list the contents of "/home/xubuntu/test", but not "/home/xubuntu/test/subtest".
> The number of open DIR handles is being monitored with the command:
> lsof +E -P -p 347813 | tr -s " " | cut -d" " -f5 | sort | uniq -c
> where 347813 is the pid of the NiFi process.
> DIR handle counts start at 4 immediately after NiFi is launched and steadily grow to
tens of thousands, until file handles are exhausted.  Additionally, these handles do not
seem to be released even if the ListFile processor is stopped.
> Please let me know if there is any additional information I can provide.

This message was sent by Atlassian Jira

View raw message