hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel
Date Fri, 11 Dec 2015 03:47:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15052109#comment-15052109
] 

Vinayakumar B edited comment on HDFS-8578 at 12/11/15 3:47 AM:
---------------------------------------------------------------

I think its a good idea to run all {{linkBlocks}} in parallel. 
But I do believe patch will not work.

Because, during {{bpStorage.recoverTransitionRead}}, {{linkBlocks}} will be made async, but
while returning, immediately, rename of current-previous happens, also it logs as upgrade
completed, even before all links are created and actual upgrade is over.

In my v15 patch, synchronization was removed on {{addStorageLocations}} because, getDatanodeUuid()
was also synchronized, and threads were getting blocked because of this.
But now, since datanodeUuId is updated only once, we can remove synchronization on this. and
can make it volatile.
Will update the patch shortly.


was (Author: vinayrpet):
I think its a good idea to run all {{linkBlocks}} in parallel. 
But I do believe patch will not work.

Because, during {{bpStorage.recoverTransitionRead}}, {{linkBlocks}} will be made async, but
while returning, immediately, rename of current-previous happens, also it logs as upgrade
completed, even before all links are created and actual upgrade is over.

In my v14 patch, synchronization was removed on {{addStorageLocations}} because, getDatanodeUuid()
was also synchronized, and threads were getting blocked because of this.
But now, since datanodeUuId is updated only once, we can remove synchronization on this. and
can make it volatile.
Will update the patch shortly.

> On upgrade, Datanode should process all storage/data dirs in parallel
> ---------------------------------------------------------------------
>
>                 Key: HDFS-8578
>                 URL: https://issues.apache.org/jira/browse/HDFS-8578
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Raju Bairishetti
>            Assignee: Vinayakumar B
>            Priority: Critical
>         Attachments: HDFS-8578-01.patch, HDFS-8578-02.patch, HDFS-8578-03.patch, HDFS-8578-04.patch,
HDFS-8578-05.patch, HDFS-8578-06.patch, HDFS-8578-07.patch, HDFS-8578-08.patch, HDFS-8578-09.patch,
HDFS-8578-10.patch, HDFS-8578-11.patch, HDFS-8578-12.patch, HDFS-8578-13.patch, HDFS-8578-14.patch,
HDFS-8578-15.patch, HDFS-8578-branch-2.6.0.patch, HDFS-8578-branch-2.7-001.patch, HDFS-8578-branch-2.7-002.patch,
HDFS-8578-branch-2.7-003.patch, h8578_20151210.patch
>
>
> Right now, during upgrades datanode is processing all the storage dirs sequentially.
Assume it takes ~20 mins to process a single storage dir then  datanode which has ~10 disks
will take around 3hours to come up.
> *BlockPoolSliceStorage.java*
> {code}
>    for (int idx = 0; idx < getNumStorageDirs(); idx++) {
>       doTransition(datanode, getStorageDir(idx), nsInfo, startOpt);
>       assert getCTime() == nsInfo.getCTime() 
>           : "Data-node and name-node CTimes must be the same.";
>     }
> {code}
> It would save lots of time during major upgrades if datanode process all storagedirs/disks
parallelly.
> Can we make datanode to process all storage dirs parallelly?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message