hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3731) 2.0 release upgrade must handle blocks being written from 1.0
Date Fri, 17 Aug 2012 23:00:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437124#comment-13437124
] 

Suresh Srinivas commented on HDFS-3731:
---------------------------------------

Colin thanks for the detailed comment. My concern is related to the durability guarantees.
Rbw was added specifically to support hsync. The implementation has changed from 1.x to 2.x.
I wanted to make sure we ensure we have both sync and lease recovery related testing to ensure
hsync files are preserved during upgrade.

I am currently busy to really look at the code and ensure the correctness of code. Others
I think who know this quite well are Todd or Nicholas. Not sure if they have time. I can get
back to this may after a week.
                
> 2.0 release upgrade must handle blocks being written from 1.0
> -------------------------------------------------------------
>
>                 Key: HDFS-3731
>                 URL: https://issues.apache.org/jira/browse/HDFS-3731
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: Suresh Srinivas
>            Assignee: Colin Patrick McCabe
>            Priority: Blocker
>         Attachments: HDFS-3731.002.patch, HDFS-3731.003.patch
>
>
> Release 2.0 upgrades must handle blocks being written to (bbw) files from 1.0 release.
Problem reported by Brahma Reddy.
> The {{DataNode}} will only have one block pool after upgrading from a 1.x release.  (This
is because in the 1.x releases, there were no block pools-- or equivalently, everything was
in the same block pool).  During the upgrade, we should hardlink the block files from the
{{blocksBeingWritten}} directory into the {{rbw}} directory of this block pool.  Similarly,
on {{-finalize}}, we should delete the {{blocksBeingWritten}} directory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message