hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Mackrory (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x
Date Tue, 10 Jan 2017 22:40:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816405#comment-15816405
] 

Sean Mackrory commented on HDFS-11096:
--------------------------------------

After looking at where SortedMapWritable is used some more, I'm more convinced it's only a
concern if we care about source compatibility, which is not required for rolling upgrades.
I also had a second look at wire compatibility and I found a few concerning things I'll look
at and possibly fix:

* The message getHdfsBlockLocations has disappeared as well as related types
* The field nonDfsUsed in DatanodeInfoProto changed from index 9 to index 15

For YARN (CC [~kasha]), the field nodeLabels in several structures in yarn_protos changed
from string to a custom type, and memory in ResourceProto change from int32 to int64 (not
sure if it's the case in protobuf, but that may not be incompatible?)

There's also a lot of messages moving between files, but not otherwise changing in any incompatible
way. That's not a concern is it?

If anyone else wants to see the changes in protobuf, this is what I did (if anything, you'll
want to replace meld with your own diff-tool-of-choice):

{quote}
#!/usr/bin/env bash

cd /tmp

OLD=branch-2.7
NEW=trunk

mkdir new
mkdir old
git clone git://git.apache.org/hadoop.git

function gather_protos() {
  SOURCE=${1}
  TARGET=${2}
  for proto in $(cd ${SOURCE} && find . -name \*.proto | sed -e 's|^\./||'); do
    #flattened=${proto//\//_} # Trips up on files that moved
    flattened=$(basename ${proto})
    cp ${SOURCE}/${proto} ${TARGET}/${flattened}
  done
}

(cd hadoop; git checkout ${OLD})
gather_protos hadoop old

(cd hadoop; git checkout ${NEW})
gather_protos hadoop new

meld old new
{quote}

> Support rolling upgrade between 2.x and 3.x
> -------------------------------------------
>
>                 Key: HDFS-11096
>                 URL: https://issues.apache.org/jira/browse/HDFS-11096
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: rolling upgrades
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Andrew Wang
>            Priority: Blocker
>
> trunk has a minimum software version of 3.0.0-alpha1. This means we can't rolling upgrade
between branch-2 and trunk.
> This is a showstopper for large deployments. Unless there are very compelling reasons
to break compatibility, let's restore the ability to rolling upgrade to 3.x releases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message