hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1792) Add code to detect valid length of an edits file
Date Tue, 05 Apr 2011 06:00:08 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015773#comment-13015773
] 

Todd Lipcon commented on HDFS-1792:
-----------------------------------

hm, that's interesting. In every case I've seen, it's always been filled with 0s.

I think this method is still "safe" in that it's just used to pick which log to recover from,
and then the usual recovery mechanism happens. So if random bytes fill the remainder of the
file, we degenerate to what we do today, and it's essentially random which one gets used for
recovery. Does that make sense?

> Add code to detect valid length of an edits file
> ------------------------------------------------
>
>                 Key: HDFS-1792
>                 URL: https://issues.apache.org/jira/browse/HDFS-1792
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: Edit log branch (HDFS-1073)
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: Edit log branch (HDFS-1073)
>
>         Attachments: hdfs-1792.txt
>
>
> In some edit log corruption situations, it's useful to be able to determine the "valid"
length of an edit log. For this JIRA we define "valid" as the length of the file excluding
any trailing 0x00 bytes, usually left there by the preallocation done while writing. In the
future this API can be extended to look at edit checksums, etc.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message