hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-2656) Support for upgrading existing cluster to facilitate appends to HDFS files
Date Thu, 10 Apr 2008 12:56:06 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

dhruba borthakur updated HADOOP-2656:
-------------------------------------

    Attachment: upgradeGenStamp4.patch

This patch does the following:

1. Upgrades existing clusters to the new disk format. Each Block has a generation stamp associated
with it. Existing blocks get a generation stamp of 0. The generation stamp is used to create
the name of the block metafile on the datanode.

2. The Block object has a new field called "generationStamp" of type "long". The BlocksMap
on the namenode is keyed on the blockid and the generation stamp.

3. The datanode sends the generation stamp, block id and size of each block in a block report.

3. All log statements that print the blockid now prints the blockid and generationstamp.

4. The client receives the blockid and generation stamp as part of an RPC that receives a
block. 

5. The DataTransferProtocol sends the blockid and generation stamp with every connection request
that the client makes to the datanode(s).


> Support for upgrading existing cluster to facilitate appends to HDFS files
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-2656
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2656
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: upgradeGenStamp4.patch
>
>
> HADOOP-1700 describes the design for supporting appends to HDFS files. This design requires
a distributed-upgrade to existing cluster installations. The design specifies that the DataNode
persist the 8-byte BlockGenerationStamp in the block metadata file. The upgrade code will
introduce this new field in the block metadata file and initialize this value to 0.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message