hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yiqun Lin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-11580) Ozone: Support asynchronus client API for SCM and containers
Date Tue, 25 Apr 2017 12:06:04 GMT

     [ https://issues.apache.org/jira/browse/HDFS-11580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Yiqun Lin updated HDFS-11580:
    Attachment: HDFS-11580-HDFS-7240.004.patch

Thanks [~anu], [~vagarychen] and [~msingh] for the great comments! All the comments make sense
to me. 
The following are my comments:
 Do you think we should have 2 functions, like readChunk and readChunkAsync so that it looks
more like java 8-ish ? rather than a boolean flag ?
Has addressed this in the latest patch.

2. The problem of that the current async calls still seem like synchronous calls. Yes, I think
this should be a problem here. As [~vagarychen] mentioned, we should not invoke {{get()}}
in the same thread. Maybe we can register a callback or other way, I take a look into {{CompletableFuture}},
there are already many APIs we can use for this case. In my latest patch, I used one APIs
named {{CompletableFuture#thenApply}} to deal with the future result asynchronous and return
the new CompletableFuture object. This should be the right way to return the CompletableFuture
object for client and let client to call future.get().

 Also because of the async nature of the interface responses, need not be in the same order
as the requests. We will need a method to match the response to the replies.
This is a good catch. If the async interface introduced, we should be more carefully to get
corresponding response of each request. In my latest patch, I defined a new map to store the
pending response. The more details can see in the method {{XceiverClientHandler#waitForResponse}}.

With an async interface, we will always need to keep an eye on the queue depth..
Good idea. But I'd like to do this work in another JIRA since current patch seems a little
big now, :).

I have changed many places in the latest patch. Any comment/suggestion are welcomed.

> Ozone: Support asynchronus client API for SCM and containers
> ------------------------------------------------------------
>                 Key: HDFS-11580
>                 URL: https://issues.apache.org/jira/browse/HDFS-11580
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>    Affects Versions: HDFS-7240
>            Reporter: Anu Engineer
>            Assignee: Yiqun Lin
>         Attachments: HDFS-11580-HDFS-7240.001.patch, HDFS-11580-HDFS-7240.002.patch,
HDFS-11580-HDFS-7240.003.patch, HDFS-11580-HDFS-7240.004.patch
> This is an umbrella JIRA that needs to support a set of APIs in Asynchronous form.
> For containers -- or the datanode API currently supports a call {{sendCommand}}. we need
to build proper programming interface and support an async interface.
> There is also a set of SCM API that clients can call, it would be nice to support Async
interface for those too.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message