hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-13767) Add msync server implementation.
Date Fri, 03 Aug 2018 22:22:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568843#comment-16568843
] 

Chen Liang edited comment on HDFS-13767 at 8/3/18 10:21 PM:
------------------------------------------------------------

Thanks for comments [~zero45].

bq. (2) A concern about the unit test...
Thanks for trying this. I think it might be related to the recent update that enabled in-progress
edit tail. Because if I disable in-progress edit tail for this test, then add the delay you
mentioned, the test will succeed, but took longer (expected). I will need to look deeper to
see how in-progress edit tail worked with this. [~xkrogen] any fast insights on this?

bq. (3) So far these patches don't actually ...
The only reason there is msync in {{NameNodeRpcServer}} is because msync is part of {{ClientProtocol}},
and NameNodeRpcServer implements it. So changing this in NameNodeRpcServer won't compile.
I guess msync is a little bit different from other RPC calls, in that from client side, it's
another RPC call like others, but from server side, all it's logic is handled in RPC layer,
not in NN layer at all.

bq. (4) I was curious about making msync pause ...
Just some thoughts of mine: it is true that there will be no deferred call in this case, but
seems to me this will also add retry msync calls, so I don't see a particular advantage here
in terms of call queue load. It seems to me it is actually the same as deferring the call,
the only difference is whether let the client send another call, or just server viewing the
existing call as a new one (by putting it back to queue).

Actually, maybe we can have a mixed strategy, we did have some discussion offline, that if
server sees itself too far behind when checking the call, don't re-queue, but just throw exception
so that client don't bother to wait and can try a different Observer. I viewed this as an
optimization. 


was (Author: vagarychen):
Thanks for comments [~zero45].

bq. (2) A concern about the unit test...
Thanks for trying this. I think it might be related to the recent update that enabled in-progress
edit tail. Because if I disable in-progress edit tail for this test, then add the delay you
mentioned, the test will succeed, but took longer (expected). I will need to look deeper to
see how in-progress edit tail worked with this. [~xkrogen] any fast insights on this?

bq. (3) So far these patches don't actually ...
The only reason there is msync in {{NameNodeRpcServer}} is because msync is part of {{ClientProtocol}},
and NameNodeRpcServer implements it. So changing this in NameNodeRpcServer won't compile.
I guess msync is a little bit different from other RPC calls, in that from client side, it's
another RPC call like others, but from server side, all it's logic is handled in RPC layer,
not in NN layer at all.

> Add msync server implementation.
> --------------------------------
>
>                 Key: HDFS-13767
>                 URL: https://issues.apache.org/jira/browse/HDFS-13767
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Chen Liang
>            Assignee: Chen Liang
>            Priority: Major
>         Attachments: HDFS-13767-HDFS-12943.001.patch, HDFS-13767-HDFS-12943.002.patch,
HDFS-13767.WIP.001.patch, HDFS-13767.WIP.002.patch, HDFS-13767.WIP.003.patch, HDFS-13767.WIP.004.patch
>
>
> This is a followup on HDFS-13688, where msync API is introduced to {{ClientProtocol}}
but the server side implementation is missing. This is Jira is to implement the server side
logic.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message