hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access
Date Thu, 10 Mar 2016 17:51:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189626#comment-15189626

Chris Nauroth commented on HDFS-9924:

bq. If you can't make any guarantees about ordering of calls in a client, then I find it really
hard to see how I would use this as a developer except for the odd cleanup operation like
an async delete, or a series of mkdir calls I knew were independent.

If you buy into my argument over on HADOOP-12910, then even without ordering guarantees, this
is helpful for use cases like mass Hive partition renames.

bq. Example, if the async operation took a function ()->T, you could have a chain of operations...

This is like futures + promises that I mentioned earlier.  I had been thinking of it as a
future (no pun intended) enhancement outside the scope of this JIRA, but perhaps it's not
too much effort to lay down the right method signatures to support it upfront.  That's worth

> [umbrella] Asynchronous HDFS Access
> -----------------------------------
>                 Key: HDFS-9924
>                 URL: https://issues.apache.org/jira/browse/HDFS-9924
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Xiaobing Zhou
> This is an umbrella JIRA for supporting Asynchronous HDFS Access.
> Currently, all the API methods are blocking calls -- the caller is blocked until the
method returns.  It is very slow if a client makes a large number of independent calls in
a single thread since each call has to wait until the previous call is finished.  It is inefficient
if a client needs to create a large number of threads to invoke the calls.
> We propose adding a new API to support asynchronous calls, i.e. the caller is not blocked.
 The methods in the new API immediately return a Java Future object.  The return value can
be obtained by the usual Future.get() method.

This message was sent by Atlassian JIRA

View raw message