hadoop-zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andraz Tori (JIRA)" <j...@apache.org>
Subject [jira] Commented: (ZOOKEEPER-107) Allow dynamic changes to server cluster membership
Date Thu, 20 May 2010 10:36:57 GMT

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12869554#action_12869554

Andraz Tori commented on ZOOKEEPER-107:

Has anything happened with this feature?

There was some talk about what the most important use cases are on the mailing list. We're
thinking of migrating home-grown solution to Zookeeper, but can't do it without dynamic addition/removal
of the servers. If it helps, here's the use case:

We're having fully cloudy solution. Every server that we put into the cluster runs a set of
services that make themselves available to a local "resource manager" that shares the list
of resources with all other servers in the cluster. When we do upgrades we simply fire up
new servers with new versions of the services and connect their resource managers to the old
ones into the same cluster. Then we simply shut down the old servers. Beside adding/removing
servers when upgrading, we also do the same thing when we need to temporarily scale - we fire
up a few more servers and connect their resource managers to the cluster to make the services
available to the cluster. 
We never know how many servers there are going to be in the cluster and we don't assign any
dns entries to them (just another point of failure).
The clients that need to know about resources connect to any of the "resource managers" and
get a list of all resources available and also about other "resource managers". As servers
move around they also can connect to different "resource manager".

This is a bit unusual configuration since cloud practically lives on its own without any kind
of static addresses. As long as you are able to connect to it at one point in time, you can
keep up with it 'motion'. 

So the idea was to migrate the above system to Zookeeper. Every service would connect to local
Zookeeper and create ephemeral node announcing it. So every server would run its own Zookeeper
node connected to the Zookeeper cloud. However without dynamic addition/removal of the servers
all this becomes infeasible.

Ideally we'd like to have a situation where we just start a Zookeeper node by giving it a
list of known other Zookeeper nodes in the cloud. And then it should take on to the life of
its own.

Hope that the use case helps. I am really looking forward to this!

> Allow dynamic changes to server cluster membership
> --------------------------------------------------
>                 Key: ZOOKEEPER-107
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-107
>             Project: Zookeeper
>          Issue Type: Improvement
>          Components: server
>            Reporter: Patrick Hunt
>            Assignee: Henry Robinson
>         Attachments: SimpleAddition.rtf
> Currently cluster membership is statically defined, adding/removing hosts to/from the
server cluster dynamically needs to be supported.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message