hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1822) Editlog opcodes overlap between 20 security and later releases
Date Fri, 08 Apr 2011 20:17:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017628#comment-13017628
] 

Suresh Srinivas commented on HDFS-1822:
---------------------------------------

20.security (LAYOUT_VERSION = -19) adds the following opcodes:
  private static final byte OP_GET_DELEGATION_TOKEN = 15; //new delegation token
  private static final byte OP_RENEW_DELEGATION_TOKEN = 16; //renew delegation token
  private static final byte OP_CANCEL_DELEGATION_TOKEN = 17; //cancel delegation token
  private static final byte OP_UPDATE_MASTER_KEY = 18; //update master key

21 (layout version = -24) adds the following opcodes:
  private static final byte OP_RENAME = 15;  // new rename
  private static final byte OP_CONCAT_DELETE = 16; // concat files.
  private static final byte OP_SYMLINK = 17; // a symbolic link
  private static final byte OP_GET_DELEGATION_TOKEN = 18; //new delegation token
  private static final byte OP_RENEW_DELEGATION_TOKEN = 19; //renew delegation token
  private static final byte OP_CANCEL_DELEGATION_TOKEN = 20; //cancel delegation token
  private static final byte OP_UPDATE_MASTER_KEY = 21; //update master key

22 (layout version = -27) and trunk (layout version > -27) adds the following opcodes:
    public static final byte OP_RENAME = 15;  // new rename
    public static final byte OP_CONCAT_DELETE = 16; // concat files.
    public static final byte OP_SYMLINK = 17; // a symbolic link
    public static final byte OP_GET_DELEGATION_TOKEN = 18; //new delegation token
    public static final byte OP_RENEW_DELEGATION_TOKEN = 19; //renew delegation token
    public static final byte OP_CANCEL_DELEGATION_TOKEN = 20; //cancel delegation token
    public static final byte OP_UPDATE_MASTER_KEY = 21; //update master key


Conflicts in the opcodes:
# Opcode 15 means OP_GET_DELEGATION_TOKEN on 20.s and OP_RENAME on later releases
# Opcode 16 means OP_RENEW_DELEGATION_TOKEN on 20.s and OP_CONCAT_DELETE on later releases
# Opcode 17 means OP_CANCEL_DELEGATION_TOKEN on 20.s and OP_SYMLINK on later releases
# Opcode 18 means OP_UPDATE_MASTER_KEY on 20.s and OP_GET_DELEGATION_TOKEN on later releases


We need to support the following upgrades:
# 20.s to 22 or later releases
#* The opcode conflict here makes consuming editlogs impossible
# Need to support upgrade from 21 to 22 or later releases


I am proposing handling these conflicts as follows, while consuming editlogs:
# If layout version is > -24 then it is 20 version, use the definition as shown in 20.security
# If layout version is <= -24 use the definition from 21 onwards.

This is messy way of doing it. But I do not see any way around it.


In future:
- We need to make sure, any op code added is added in the trunk first, before adding it in
older releases. Trunk should be the source of truth, ensuring the opcodes are chosen uniquely
across different releases.


> Editlog opcodes overlap between 20 security and later releases
> --------------------------------------------------------------
>
>                 Key: HDFS-1822
>                 URL: https://issues.apache.org/jira/browse/HDFS-1822
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.21.0, 0.22.0, 0.23.0
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>            Priority: Blocker
>             Fix For: 0.22.0, 0.23.0
>
>
> Same opcode are used for different operations between 0.20.security, 0.22 and 0.23. This
results in failure to load editlogs on later release, especially during upgrades.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message