hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1801) Remove use of timestamps to identify checkpoints and logs
Date Sat, 02 Apr 2011 17:34:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015056#comment-13015056

stack commented on HDFS-1801:

Would a txid ever be available in this backup context?

-    storage.setCheckpointTime(sig.checkpointTime);
+    // storage.setCheckpointTime(sig.checkpointTime);
+    // TODO do something with checkpoint txid?

(I see it in the FSImage context)

Why you do Preconditions sometimes and assert elsewhere?  I'd think you'd do one or the other?

Will we need tests where we check migration, where we prove the new code can open the old-style
fsimages and edit files?

The txid is a long as was the ts so Serialization doesn't change; the number of bytes doesn't
change, just their interpretation?  Is it possible that we'll deserialize an old style NameNodeRegistration
and interpret a ts as a txid?

Patch looks good.

> Remove use of timestamps to identify checkpoints and logs
> ---------------------------------------------------------
>                 Key: HDFS-1801
>                 URL: https://issues.apache.org/jira/browse/HDFS-1801
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: Edit log branch (HDFS-1073)
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: Edit log branch (HDFS-1073)
>         Attachments: hdfs-1801.txt
> Currently, the NameNode validates checkpoint uploads by using timestamps associated with
checkpoints and edit logs. However, now that we have transaction IDs that uniquely identify
each point in time in the history of a namespace, it is more robust to simply use transaction
IDs to identify images and edits.
> This JIRA is to remove the use of editsTime and checkpointTime and replace it with:
> * {{lastCheckpointTxId}} - the highest transaction ID reflected in the most recently
saved fsimage file
> * {{lastLogRollTxId}} - the highest transaction ID in {{edits}} when {{rollFsImage}}
was called by the checkpointing node.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message