hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11580) Ozone: Support asynchronus client API for SCM and containers
Date Wed, 26 Jul 2017 17:47:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101998#comment-16101998
] 

Chen Liang commented on HDFS-11580:
-----------------------------------

Thanks [~nandakumar131] for the comments!

As far as I remember how this JIRA has come along, the potential issue here was that it was
not clear whether the response will match the request. For example, say, the async readfile
is calling readfilesync directly, i.e.
{code}
Data readFileAsync(..) {
  return readFileSync(...)
}
{code}
And we are calling readSmallFile twice, to read file1 and file2.
{code}
readFileAsync(file1)
readFileAsync(file2)
{code}
There is no guarantee which request arrives and returns first. So the readFileSync(file2)
can potentially return earlier, if so, can we be sure it's returned gets delivered to second
call definitely, instead of the first? Essentially, the issue is, if there are two simultaneous
calls of {{supplyAsync}} with the same supplier, and the supplier returns one time, which
of the two {{supplyAsync}} will return?

This is related to {{CompletableFuture}}'s feature I haven't digged into or tested. Totally
possible that it be will handled well in some way. If you (or anyone looking at this) have
anything to add, please feel free to share. But as I follow this JIRA, it was because of this,
the implementation is that {{sendCommandAsync}} is calling {{waitForResponse}}, which has
an internal map that matches the response to the request. 

 

> Ozone: Support asynchronus client API for SCM and containers
> ------------------------------------------------------------
>
>                 Key: HDFS-11580
>                 URL: https://issues.apache.org/jira/browse/HDFS-11580
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>    Affects Versions: HDFS-7240
>            Reporter: Anu Engineer
>            Assignee: Yiqun Lin
>         Attachments: HDFS-11580-HDFS-7240.001.patch, HDFS-11580-HDFS-7240.002.patch,
HDFS-11580-HDFS-7240.003.patch, HDFS-11580-HDFS-7240.004.patch, HDFS-11580-HDFS-7240.005.patch,
HDFS-11580-HDFS-7240.006.patch, HDFS-11580-HDFS-7240.007.patch, HDFS-11580-HDFS-7240.008.patch,
HDFS-11580-HDFS-7240.009.patch
>
>
> This is an umbrella JIRA that needs to support a set of APIs in Asynchronous form.
> For containers -- or the datanode API currently supports a call {{sendCommand}}. we need
to build proper programming interface and support an async interface.
> There is also a set of SCM API that clients can call, it would be nice to support Async
interface for those too.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message