hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access
Date Thu, 09 Jun 2016 17:03:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322864#comment-15322864

stack commented on HDFS-9924:

bq. This JIRA and the current implementation originally aim to support a basic async access
to HDFS without callback support and without chaining support.

The JIRA is about HDFS async access. There is no exception in the summary nor description
to rule out the basic async callback primitive. You could rule it out via fiat -- can you
even call it an 'async' API if it doesn't do callback? -- but why not do it right from the
get-go. Do it once only too.

bq. When we change to return XxxFuture in the future, it is a backward compatible change.

You and [~jnp] have said this a few times but for downstreamers, a Future-only API is not
worth engaging with. It means each of us has to build parking structures to keep the unfinished
the Futures in, polling to look for completions to react too. This is a performance-killer.
Been there. Done that.

I like the [~mingma] summary/suggestion with the [~andrew.wang] caveat; revert and dev in
a feature branch against trunk. I know of a few downstreamers that are interested, myself
included, and would be up for helping out. Thanks.

> [umbrella] Asynchronous HDFS Access
> -----------------------------------
>                 Key: HDFS-9924
>                 URL: https://issues.apache.org/jira/browse/HDFS-9924
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Xiaobing Zhou
>         Attachments: AsyncHdfs20160510.pdf
> This is an umbrella JIRA for supporting Asynchronous HDFS Access.
> Currently, all the API methods are blocking calls -- the caller is blocked until the
method returns.  It is very slow if a client makes a large number of independent calls in
a single thread since each call has to wait until the previous call is finished.  It is inefficient
if a client needs to create a large number of threads to invoke the calls.
> We propose adding a new API to support asynchronous calls, i.e. the caller is not blocked.
 The methods in the new API immediately return a Java Future object.  The return value can
be obtained by the usual Future.get() method.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message