hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2085) 1073: finalize inprogress edit logs at startup
Date Tue, 21 Jun 2011 18:41:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13052755#comment-13052755
] 

Todd Lipcon commented on HDFS-2085:
-----------------------------------

That's an interesting point... there's not currently any test that checks for that, and in
fact END_LOG_SEGMENT is basically a placeholder at this point. The idea is that in the future
it might be used to contain some statistics about what's in the log, etc.

On the pro side, it would be nice to do this to make it an invariant that finalized log files
include END_LOG_SEGMENT. On the con side, it involves reopening a log for append which isn't
an operation we currently do anywhere else.

Mind if we open this as another JIRA since it doesn't currently cause any issues that I can
think of?

> 1073: finalize inprogress edit logs at startup
> ----------------------------------------------
>
>                 Key: HDFS-2085
>                 URL: https://issues.apache.org/jira/browse/HDFS-2085
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: Edit log branch (HDFS-1073)
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: Edit log branch (HDFS-1073)
>
>         Attachments: hdfs-2085.txt
>
>
> With HDFS-2074, the NameNode can read through any "in-progress" logs it finds during
startup to determine how many transactions they have. It can then re-name the file from its
inprogress name to its finalized name. For example, if it finds a file edits_10_inprogress
with 3 transactions, it can rename it to edits_10-12 at startup. This means that other parts
of the system like edits-log-transfer don't need to worry about in-progress logs.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message