hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6038) JournalNode hardcodes NameNodeLayoutVersion in the edit log file
Date Thu, 06 Mar 2014 22:45:44 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923198#comment-13923198
] 

Tsz Wo Nicholas Sze commented on HDFS-6038:
-------------------------------------------

- DataOutputBuffer.writeInt can be simplified as below.
{code}
//DataOutputBuffer.java  
  public void writeInt(int v, int offset) throws IOException {
    int oldCount = buffer.setOffset(offset);
    writeInt(v);
    buffer.setOffset(oldCount);
  }
{code}

- BookKeeperEditLogInputStream.scanNextOp() can be removed.

- In EditLogFileInputStream.scanEditLog, the txid variable can be declared inside the while-loop.

- FSEditLogOp.Reader.scanOp() needs to check OP_INVALID like decodeOp().


> JournalNode hardcodes NameNodeLayoutVersion in the edit log file
> ----------------------------------------------------------------
>
>                 Key: HDFS-6038
>                 URL: https://issues.apache.org/jira/browse/HDFS-6038
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, ha, hdfs-client, namenode
>            Reporter: Haohui Mai
>            Assignee: Jing Zhao
>         Attachments: HDFS-6038.000.patch, HDFS-6038.001.patch, HDFS-6038.002.patch, HDFS-6038.003.patch,
HDFS-6038.004.patch, editsStored
>
>
> In HA setup, the JNs receive edit logs (blob) from the NN and write into edit log files.
In order to write well-formed edit log files, the JNs prepend a header for each edit log file.
> The problem is that the JN hard-codes the version (i.e., {{NameNodeLayoutVersion}} in
the edit log, therefore it generates incorrect edit logs when the newer release bumps the
{{NameNodeLayoutVersion}} during rolling upgrade.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message