hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jitendra Nath Pandey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access
Date Tue, 07 Jun 2016 05:05:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317858#comment-15317858

Jitendra Nath Pandey commented on HDFS-9924:

The features are not played or experimented with if they are in a feature branch. No one will
use if it is not part of a release, or at least in a release branch or trunk.
The best way to actually harden an API is to have downstream users use them and provide feedback.
@Unstable was defined for exactly the same purpose. 
  As suggested by [~steve_l] earlier, if it helps an additional annotation @Experimental can
be created to more strongly highlight the unstable/experimental nature of the API. 
  The current implementation is not a user facing API therefore it will have to be modified
when final API is decided in HADOOP-12910.

bq. ... figuring a general approach for async'ing the Filesystem.
There is a proposal on HADOOP-12910 and lets have a discussion around that, to ensure right
API gets committed. 

bq ... Retrofitting callback on Future,
I agree it is not a good idea. I think CompletableFuture in jdk8 seems like a good choice.
I am leaning to support CompletableFuture in trunk, and if its too hard to mimic in branch-2,
don't expose a user facing API in branch-2 at all.
Users in branch-2 can use the current unstable/experimental version in {{DistributedFileSystem}}
to try out. Since CompletableFuture implements Future, it will still not be incompatible in
trunk, although we don't guarantee that. 

> [umbrella] Asynchronous HDFS Access
> -----------------------------------
>                 Key: HDFS-9924
>                 URL: https://issues.apache.org/jira/browse/HDFS-9924
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Xiaobing Zhou
>         Attachments: AsyncHdfs20160510.pdf
> This is an umbrella JIRA for supporting Asynchronous HDFS Access.
> Currently, all the API methods are blocking calls -- the caller is blocked until the
method returns.  It is very slow if a client makes a large number of independent calls in
a single thread since each call has to wait until the previous call is finished.  It is inefficient
if a client needs to create a large number of threads to invoke the calls.
> We propose adding a new API to support asynchronous calls, i.e. the caller is not blocked.
 The methods in the new API immediately return a Java Future object.  The return value can
be obtained by the usual Future.get() method.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message