nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Holger Frydrych (JIRA)" <j...@apache.org>
Subject [jira] [Created] (NIFI-4434) ListHDFS applies File Filter also to subdirectory names in recursive search
Date Thu, 28 Sep 2017 13:15:00 GMT
Holger Frydrych created NIFI-4434:
-------------------------------------

             Summary: ListHDFS applies File Filter also to subdirectory names in recursive
search
                 Key: NIFI-4434
                 URL: https://issues.apache.org/jira/browse/NIFI-4434
             Project: Apache NiFi
          Issue Type: Bug
    Affects Versions: 1.3.0
            Reporter: Holger Frydrych


The File Filter regex configured in the ListHDFS processor is applied not just to files found,
but also to subdirectories. 

If you try to set up a recursive search to list e.g. all csv files in a directory hierarchy
via a regex like ".*\.csv", it will only pick up csv files in the base directory, not in any
subdirectory. This is because subdirectories don't typically match that regex pattern.

To fix this, either subdirectories should not be matched against the file filter, or the file
filter should be applied to the full path of all files (relative to the base directory). The
GetHDFS processor offers both options via a switch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message