hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-624) Client support pipeline recovery
Date Tue, 22 Sep 2009 23:56:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758500#action_12758500
] 

Tsz Wo (Nicholas), SZE commented on HDFS-624:
---------------------------------------------

Have some comments on FSDataset:

- In FSDataset.append(..), should we check whether newGS > replicaInfo's gs?

- Not sure if you want to fix it: The terms "block" and "replica" should be used with care.
 In datanode, "replica" probably is a better term most of cases.  Some examples:
-* BlockNotFoundException should be ReplicaNotFoundException
-* throw new IOException("Block " + replicaInfo + " reopen failed. " + ...

- renameTo(..) in the following may throw IOException.  In such case, the old gs should be
restored.  It is better to check the oldmeta file since renameTo(..) may not be atomic and
the oldmeta file may not exist.
{code}
+    if (!oldmeta.renameTo(newmeta)) {
+      replicaInfo.setGenerationStamp(oldGS); // restore old GS
+      throw new IOException("Block " + (Block)replicaInfo + " reopen failed. " +
+                            " Unable to move meta file  " + oldmeta +
+                            " to " + newmeta);
+    }
{code}


> Client support pipeline recovery
> --------------------------------
>
>                 Key: HDFS-624
>                 URL: https://issues.apache.org/jira/browse/HDFS-624
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: Append Branch
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: Append Branch
>
>         Attachments: pipelineRecovery.patch, pipelineRecovery1.patch
>
>
> This jira aims to
> 1. set up initial pipeline for append;
> 2. recover failed pipeline setup for append;
> 2. set up pipeline to recover failed data streaming.
> The algorithm is described in the design document in the pipeline recovery and pipeline
set up sections. Pipeline close and failed pipeline close are not included in this jira. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message