Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 27 Oct 2017 20:40:00 +0000 (UTC)
From: "Sean Mackrory (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13017442.1478127231000.82294.1509136800753@Atlassian.JIRA>
In-Reply-To: <JIRA.13017442.1478127231000@Atlassian.JIRA>
References: <JIRA.13017442.1478127231000@Atlassian.JIRA> <JIRA.13017442.1478127231818@jira-lw-us.apache.org>
Subject: [jira] [Updated] (HDFS-11096) Support rolling upgrade between 2.x
 and 3.x
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 27 Oct 2017 20:40:07 -0000


     [ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Mackrory updated HDFS-11096:
---------------------------------
    Attachment: HDFS-11096.007.patch

Okay - I'm now pretty happy with how this is working. I saw the last shellcheck problems locally, and have fixed those, too. I've had several successful test runs of both of the Docker tests in the last few days, and this is looking pretty reliable to me:

* Versions to test are now specified via CLI args to the Docker scripts. That way this only has to change in code when there's a bug to fix or other improvement to make: Jenkins jobs can be updated for various version combinations independently.
* Fixing more ZK timeouts, this time in YARN. I've disabled the YARN rolling upgrade as that appears to be troublesome again. But the HDFS upgrade is working and YARN / MR is working well during and after that upgrade. I'll keep troubleshooting the YARN side, but that can be a separate JIRA.
* Logs are now saved to ./logs/ back on the host to facilitate more debugging after the Docker images have been destroyed in the event of a failure.

Although I've made a number of fixes as documented in the comments, not much has changed that would invalidate the value of previous code reviews, IMO. [~aw] - have I addressed the issues you pointed out to your satisfaction?

> Support rolling upgrade between 2.x and 3.x
> -------------------------------------------
>
>                 Key: HDFS-11096
>                 URL: https://issues.apache.org/jira/browse/HDFS-11096
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: rolling upgrades
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Andrew Wang
>            Assignee: Sean Mackrory
>            Priority: Blocker
>         Attachments: HDFS-11096.001.patch, HDFS-11096.002.patch, HDFS-11096.003.patch, HDFS-11096.004.patch, HDFS-11096.005.patch, HDFS-11096.006.patch, HDFS-11096.007.patch
>
>
> trunk has a minimum software version of 3.0.0-alpha1. This means we can't rolling upgrade between branch-2 and trunk.
> This is a showstopper for large deployments. Unless there are very compelling reasons to break compatibility, let's restore the ability to rolling upgrade to 3.x releases.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org