helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandeep Nayak <osgig...@gmail.com>
Subject Re: Helix 0.6.2- Messages on startup
Date Thu, 20 Feb 2014 21:48:08 GMT
So let me take a few steps back and put forth a use-case which might
throw some light on what I am pursuing.

Lets say I have 3 nodes A, B and C and I want to create a peer-peer
cluster so that all of them join a cluster (like the jgroups model). I
then want to add resources to A or B or C and expect the specific node
to push the resource information to the cluster based on the
resource's replication strategy. So lets say X is installed on A and X
states I want to be available on exactly 2 nodes then I imagine the
cluster will pick two of A, B or C to hold the resource.

So a few questions, please bear with my ignorance

* In this use-case a good fit for Helix, I would love to get the
replication strategy through Helix but if I am dealing with a inherent
master-slave model which is not peer-peer am I using Helix for the
wrong thing?
* Do all install requests get routed to the Master and then pushed to
the relevant slaves?

Thanks,

Sandeep

On Thu, Feb 20, 2014 at 1:07 PM, Kanak Biscuitwala <kanak.b@hotmail.com> wrote:
> Are you talking about the controller or the participants (instances)? If you would like
to see if the controller is running, check under /clusterName/CONTROLLER to see if there is
a ZNode called LEADER.
>
> For participants, nothing will be assigned until resources are defined. Ideal states
and external views are created and used on a per-resource level. Helix can manage multiple
distributed resources at once, so you need to add a resource to define things like the state
model it follows, the assignment policy (rebalance mode), and number of partitions and replicas.
This step generates the ideal state for the resource. The quickstart and tutorial detail these
steps.
>
> Kanak
> ----------------------------------------
>> Date: Thu, 20 Feb 2014 12:53:08 -0800
>> Subject: Re: Helix 0.6.2- Messages on startup
>> From: osgigeek@gmail.com
>> To: user@helix.apache.org
>>
>> Thanks Jason for the pointers. I do see the nodes and InstanceConfigs
>> but there is no indication of which is master, do I need to add
>> resources to the cluster for that information to surface? If nodes are
>> added to the cluster should I be able to see which is the master or
>> leader and which are the followers?
>>
>> Sandeep
>>
>> On Thu, Feb 20, 2014 at 11:51 AM, Zhen Zhang <zzhang@linkedin.com> wrote:
>>> For checking external-view, you can use zoo inspector:
>>> http://helix.apache.org/0.7.0-incubating-docs/Quickstart.html
>>>
>>> The zookeeper path is for the external view is
>>> /{clusterName}/{externalView}/{resourceName}
>>>
>>> If you want do it programmatically, you can use spectator:
>>> http://helix.apache.org/0.7.0-incubating-docs/tutorial_spectator.html
>>>
>>>
>>> The time for a node to transit to its target state depends on how long the
>>> state transition takes.
>>>
>>> In normal case, if the external-view matches the ideal-state, it indicates
>>> the cluster converges.
>>>
>>> Thanks,
>>> Jason
>>>
>>> On 2/20/14 11:29 AM, "Sandeep Nayak" <osgigeek@gmail.com> wrote:
>>>
>>>>I probably missed it in the documentation, but how can I check the
>>>>external view? Also, is there a time period after which the first node
>>>>logs that it has identified itself as the leader? Just wanted to know
>>>>what is the indicator that things work and I do have a leader and
>>>>follower in the cluster?
>>>>
>>>>Thanks,
>>>>
>>>>Sandeep
>>>>
>>>>On Thu, Feb 20, 2014 at 9:34 AM, Kanak Biscuitwala <kanak.b@hotmail.com>
>>>>wrote:
>>>>> We didn't, but now we do.
>>>>> ________________________________
>>>>>> Date: Thu, 20 Feb 2014 09:26:10 -0800
>>>>>> Subject: Re: Helix 0.6.2- Messages on startup
>>>>>> From: g.kishore@gmail.com
>>>>>> To: user@helix.apache.org
>>>>>>
>>>>>> Do we have a jira for this ?
>>>>>>
>>>>>>
>>>>>> On Wed, Feb 19, 2014 at 11:20 PM, Kanak Biscuitwala
>>>>>> <kanak.b@hotmail.com<mailto:kanak.b@hotmail.com>> wrote:
>>>>>> Hi Sandeep,
>>>>>>
>>>>>> Does Helix still generate an external view correctly despite these
log
>>>>>> messages?
>>>>>>
>>>>>> I believe the first message is a known issue: the callbacks are
>>>>>> initialized once by the HelixManager implementation, and then
>>>>>> initialized again when the controller becomes leader (for the first
>>>>>> controller, these will happen very close to one another). It's
>>>>>> generally a harmless message.
>>>>>>
>>>>>> The second one indicates that the controller is getting events (which
>>>>>> path was changed to trigger the event?), but hasn't yet become leader.
>>>>>> When it does, it will invoke the pipeline and do the right thing.
>>>>>>
>>>>>> I think in either case, a race is causing the log messages to appear,
>>>>>> but it shouldn't affect the correctness of the Helix controller
>>>>>> operation.
>>>>>>
>>>>>> Kanak
>>>>>>
>>>>>> Date: Wed, 19 Feb 2014 23:13:19 -0800
>>>>>> Subject: Helix 0.6.2- Messages on startup
>>>>>> From: osgigeek@gmail.com<mailto:osgigeek@gmail.com>
>>>>>> To: user@helix.apache.org<mailto:user@helix.apache.org>
>>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>> So I finally got my setup done with states etc. But when I start
the
>>>>>> first node I see the following messages
>>>>>>
>>>>>> expected types: [CALLBACK, FINALIZE] but was INIT
>>>>>> I see this for /cluster/Controller,
>>>>>> /cluster/INSTANCES/Instance/MESSAGES and several other paths.
>>>>>> The other message I see is
>>>>>> Cluster manager: first-node is not leader. Pipeline will not be invoked
>>>>>> Any ideas on what I am missing here?
>>>>>> Thanks,
>>>>>> Sandeep
>>>>>>
>>>

Mime
View raw message