hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesse Yates (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6440) Support more than 2 NameNodes
Date Wed, 06 May 2015 19:25:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531238#comment-14531238

Jesse Yates commented on HDFS-6440:

[~atm] thanks for the feedback. I'm working on rebasing on trunk and addressing your comments
(hopefully a patch by tomorrow), but a couple of comments/questions first:

bq. Rolling upgrades/downgrades/rollbacks.

I'm not sure how we would test this when needing to change the structure of the FS to support
more than 2 NNs. Would you recommend (1) recognizing the old layout and then (2) transfering
it into the new layout? The reason this seems silly (to me) is that the layout is only enforced
by the way the minicluster is used/setup, rather than the way things would actually be run.
By moving things into the appropriate directories per-nn, but keeping everything else below
that the same, I think we keep the same upgrade properties but don't need to do the above
contrived/synthetic "upgrade".

bq. What's a "fresh cluster" vs. a "running cluster" in this sense?

Maybe some salesforce terminology leak here. "Fresh" would be one where you just formatted
the primary NN and are bootstrapping the other NNs from that layout. "Running" would be when
bringing up a SNN after some sort of failure and it has an unformatted fs - then it can pull
from any node in the cluster. As an SNN it would then be able to catch up by tailing the ANN.

I'll update the comment.

bq. is changing the value of FAILOVER_SEED going to do anything, given that it's only ever
read at the static initialization of the failoverRandom?

Yes, it for when there is an error and you want to run the exact sequence of failovers again
in the test. Minor helper, but can be useful when trying to track down ordering dependency
issues (which there shoudn't be, but sometimes these things can creep in).

Otherwise, everything else seems completely reasonable. Thanks!

> Support more than 2 NameNodes
> -----------------------------
>                 Key: HDFS-6440
>                 URL: https://issues.apache.org/jira/browse/HDFS-6440
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: auto-failover, ha, namenode
>    Affects Versions: 2.4.0
>            Reporter: Jesse Yates
>            Assignee: Jesse Yates
>         Attachments: Multiple-Standby-NameNodes_V1.pdf, hdfs-6440-cdh-4.5-full.patch,
hdfs-6440-trunk-v1.patch, hdfs-6440-trunk-v1.patch, hdfs-multiple-snn-trunk-v0.patch
> Most of the work is already done to support more than 2 NameNodes (one active, one standby).
This would be the last bit to support running multiple _standby_ NameNodes; one of the standbys
should be available for fail-over.
> Mostly, this is a matter of updating how we parse configurations, some complexity around
managing the checkpointing, and updating a whole lot of tests.

This message was sent by Atlassian JIRA

View raw message