hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anatoli Shein (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-10679) libhdfs++: Implement parallel find with wildcards tool
Date Fri, 22 Jul 2016 16:26:20 GMT
Anatoli Shein created HDFS-10679:
------------------------------------

             Summary: libhdfs++: Implement parallel find with wildcards tool
                 Key: HDFS-10679
                 URL: https://issues.apache.org/jira/browse/HDFS-10679
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Anatoli Shein


The find tool will issue the GetListing namenode operation on a given directory, and filter
the results using posix globbing library.

If the recursive option is selected, for each returned entry that is a directory the tool
will issue another asynchronous call GetListing and repeat the result processing in a recursive
fashion.

One implementation issue that needs to be addressed is the way how results are returned back
to the user: we can either buffer the results and return them to the user in bulk, or we can
return results continuously as they arrive. While buffering would be an easier solution, returning
results as they arrive would be more beneficial to the user in terms of performance, since
the result processing can start as soon as the first results arrive without any delay. In
order to do that we need the user to use a loop to process arriving results, and we need to
send a special message back to the user when the search is over.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message