hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4300) TransferFsImage.downloadEditsToStorage should use a tmp file for destination
Date Fri, 10 May 2013 21:05:17 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Aaron T. Myers updated HDFS-4300:
---------------------------------

    Attachment: hdfs-4300-4.patch

The most recent patch doesn't apply cleanly because of the commit of HDFS-4610. Here's a rebased
version of the patch that's otherwise unchanged.

The latest patch looks pretty good to me. Todd, does it look OK to you as well? If so, I'll
go ahead and commit this rebased patch once Jenkins come back clean.
                
> TransferFsImage.downloadEditsToStorage should use a tmp file for destination
> ----------------------------------------------------------------------------
>
>                 Key: HDFS-4300
>                 URL: https://issues.apache.org/jira/browse/HDFS-4300
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.2-alpha
>            Reporter: Todd Lipcon
>            Assignee: Andrew Wang
>            Priority: Critical
>         Attachments: hdfs-4300-1.patch, hdfs-4300-2.patch, hdfs-4300-3.patch, hdfs-4300-4.patch
>
>
> Currently, in TransferFsImage.downloadEditsToStorage, we download the edits file directly
to its finalized path. So, if the transfer fails in the middle, a half-written file is left
and cannot be distinguished from a correct file. So, future checkpoints by the 2NN will fail,
since the file is truncated in the middle -- but it won't ever download a good copy because
it thinks it already has the proper file.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message