hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5138) Support HDFS upgrade in HA
Date Tue, 14 Jan 2014 05:46:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13870417#comment-13870417

Aaron T. Myers commented on HDFS-5138:

Thanks for the comments, Konst.

bq. This is less intuitive than the current state of the art. Because after NN rollback you
need to start NameNode as -regular, while DataNodes with -rollback startup option.

It's different, but not obvious to me that it's necessary less intuitive. I've personally
always found it a bit strange that to roll back you need to start the NN _once_ with the '-rollback'
option, which will result in it doing some things at startup, and then starting up as normal.
This might seem to imply that the NN is running in some sort of "rollback mode", when in fact
the act of rolling back has already completed, and thereafter you should always start the
NN without the '-rollback' option.

bq. Also just mentioning there could be some collisions with the rolling upgrade design, which
I just finished reading. I think HDFS-5535 assumes current (pre-your-patch) behaviours of
-rollback and -finalize. For -finalize the problem could be that you remove it as a start
up option. May be Suresh can elaborate better on this.

Needing to roll back should (hopefully!) be such a rare occurrence that it doesn't seem unreasonable
to me to not do that in a rolling way. Removal of the '-finalize' startup option, I would
think, should make the whole thing easier, and doesn't seem to me to have any benefits vs.
just using the finalizeUpgrade RPC.

> Support HDFS upgrade in HA
> --------------------------
>                 Key: HDFS-5138
>                 URL: https://issues.apache.org/jira/browse/HDFS-5138
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.1.1-beta
>            Reporter: Kihwal Lee
>            Assignee: Aaron T. Myers
>            Priority: Blocker
>         Attachments: HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch,
HDFS-5138.patch, HDFS-5138.patch, HDFS-5138.patch
> With HA enabled, NN wo't start with "-upgrade". Since there has been a layout version
change between 2.0.x and 2.1.x, starting NN in upgrade mode was necessary when deploying 2.1.x
to an existing 2.0.x cluster. But the only way to get around this was to disable HA and upgrade.

> The NN and the cluster cannot be flipped back to HA until the upgrade is finalized. If
HA is disabled only on NN for layout upgrade and HA is turned back on without involving DNs,
things will work, but finaliizeUpgrade won't work (the NN is in HA and it cannot be in upgrade
mode) and DN's upgrade snapshots won't get removed.
> We will need a different ways of doing layout upgrade and upgrade snapshot.  I am marking
this as a 2.1.1-beta blocker based on feedback from others.  If there is a reasonable workaround
that does not increase maintenance window greatly, we can lower its priority from blocker
to critical.

This message was sent by Atlassian JIRA

View raw message