hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yiqun Lin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11580) Ozone: Support asynchronus client API for SCM and containers
Date Wed, 26 Apr 2017 08:32:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15984356#comment-15984356
] 

Yiqun Lin edited comment on HDFS-11580 at 4/26/17 8:31 AM:
-----------------------------------------------------------

Thanks [~vagarychen], [~msingh] for the quick review! 

{quote}
will supplyAsync be called by multiple threads at the same time? how about waitForResponse?
If so, is this method thread-safe?
{quote}
Each supplyAsync call will be running in a thread pool inside. So there is a chance that multiple
threads will do the get/remove operations in pending map. So I used {{ConcurrentHashMap}}
here and it can ensure this method thread-safe.
{quote}
This can be somewhat tricky, but can we add a unit test to verify this?
{quote}
Had addressed this in the latest patch.
{quote}
 Make this one line?
{quote}
Fixed.
{quote}
Let's either file another JIRA to follow up, or throw an UnsupportedOperationException..
{quote}
I think here we would be better to throw an UnsupportedOperationException.

{quote}
However, with Cblock, there are cases where TraceID is not set correctly. I feel that TraceID
should not be used to match a response to its corresponding request. Would a counter be a
better parameter to use here ?
{quote}
I'm not so familiar with Cblock currently. But I think the way of using a counter is also
a optional way. We can use the existing class {{SequentialNumber}} to generate the unique
id. In addition, I see {{SequentialNumber}} has already used to generated blockId and blockgroupId
in HDFS. Before doing this, I suppose that we need more discussion on this.

Attach the new patch to mainly address [~vagarychen]'s comments.


was (Author: linyiqun):
Thanks [~vagarychen], [~msingh] for the quick review! 

{quote}
will supplyAsync be called by multiple threads at the same time? how about waitForResponse?
If so, is this method thread-safe?
{quote}
Each supplyAsync call will be running in a thread pool inside. So there is a chance that multiple
threads will do the get/remove operations in pending map. So I used {{ConcurrentHashMap}}
here and it can ensure this method thread-safe.
{quote}
This can be somewhat tricky, but can we add a unit test to verify this?
{quote}
Had addressed this in the latest patch.
{quote}
 Make this one line?
{quote}
Fixed.
{quote}
Let's either file another JIRA to follow up, or throw an UnsupportedOperationException..
{quote}
I think here we would be better to throw an UnsupportedOperationException.

{quote}
However, with Cblock, there are cases where TraceID is not set correctly. I feel that TraceID
should not be used to match a response to its corresponding request. Would a counter be a
better parameter to use here ?
{quote}
I'm not so familiar with Cblock currently. But I think the way of using a counter is also
a optional way. We can use the existing class {{SequentialNumber}} to generate the unique
id. In addition, I see {{SequentialNumber}} has already used to generated blockId and blockgroupId
in HDFS.

Attach the new patch to mainly address [~vagarychen]'s comments.

> Ozone: Support asynchronus client API for SCM and containers
> ------------------------------------------------------------
>
>                 Key: HDFS-11580
>                 URL: https://issues.apache.org/jira/browse/HDFS-11580
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>    Affects Versions: HDFS-7240
>            Reporter: Anu Engineer
>            Assignee: Yiqun Lin
>         Attachments: HDFS-11580-HDFS-7240.001.patch, HDFS-11580-HDFS-7240.002.patch,
HDFS-11580-HDFS-7240.003.patch, HDFS-11580-HDFS-7240.004.patch, HDFS-11580-HDFS-7240.005.patch
>
>
> This is an umbrella JIRA that needs to support a set of APIs in Asynchronous form.
> For containers -- or the datanode API currently supports a call {{sendCommand}}. we need
to build proper programming interface and support an async interface.
> There is also a set of SCM API that clients can call, it would be nice to support Async
interface for those too.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message