hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access
Date Mon, 06 Jun 2016 13:38:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316494#comment-15316494

Allen Wittenauer commented on HDFS-9924:

bq. We did not need a branch here since, as mentioned before, this change was adding mostly
new code but not changing existing code much. Therefore, this feature won't hurt stability.

It's a new feature with a significantly large API surface area. It should absolutely require
extra scrutiny before going in, new code or not.

bq. You might be upset by other features such as HDFS symlink.

I was actually thinking of some of the crazy things that are going on in YARN. 

bq.  I believe the PMC is already closely watching the commits.

... and you'd be very wrong.  Some of the things getting added with no resistance is just
amazing to me.  (e.g., "let's destroy the NN box by writing metrics to a log outside the metrics
system using config parameters that don't match anything else in HDFS" is my current favorite.)

> [umbrella] Asynchronous HDFS Access
> -----------------------------------
>                 Key: HDFS-9924
>                 URL: https://issues.apache.org/jira/browse/HDFS-9924
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Xiaobing Zhou
>         Attachments: AsyncHdfs20160510.pdf
> This is an umbrella JIRA for supporting Asynchronous HDFS Access.
> Currently, all the API methods are blocking calls -- the caller is blocked until the
method returns.  It is very slow if a client makes a large number of independent calls in
a single thread since each call has to wait until the previous call is finished.  It is inefficient
if a client needs to create a large number of threads to invoke the calls.
> We propose adding a new API to support asynchronous calls, i.e. the caller is not blocked.
 The methods in the new API immediately return a Java Future object.  The return value can
be obtained by the usual Future.get() method.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message