hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6038) JournalNode hardcodes NameNodeLayoutVersion in the edit log file
Date Thu, 06 Mar 2014 02:01:48 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13921881#comment-13921881
] 

Tsz Wo Nicholas Sze commented on HDFS-6038:
-------------------------------------------

- It may be easier to implement DataOutputBuffer.writeInt(..) if we add Buffer.setCount(..),
i.e. get old count, set new count, writeInt and then update count.
- FSEditLogOp.scanOp(), it should use in.skip(..) instead of readFully(..).

- EditLogInputStream.scanNextOp() could have a default implementation -- it just call readOp
and return txid or throw UnsupportedOperationException.  Only EditLogFileInputStream override
it and the other subclasses don't.

> JournalNode hardcodes NameNodeLayoutVersion in the edit log file
> ----------------------------------------------------------------
>
>                 Key: HDFS-6038
>                 URL: https://issues.apache.org/jira/browse/HDFS-6038
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, ha, hdfs-client, namenode
>            Reporter: Haohui Mai
>            Assignee: Jing Zhao
>         Attachments: HDFS-6038.000.patch, HDFS-6038.001.patch, HDFS-6038.002.patch, HDFS-6038.003.patch,
editsStored
>
>
> In HA setup, the JNs receive edit logs (blob) from the NN and write into edit log files.
In order to write well-formed edit log files, the JNs prepend a header for each edit log file.
> The problem is that the JN hard-codes the version (i.e., {{NameNodeLayoutVersion}} in
the edit log, therefore it generates incorrect edit logs when the newer release bumps the
{{NameNodeLayoutVersion}} during rolling upgrade.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message