helix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kisho...@apache.org
Subject [1/2] Applying patch by rms(bob) to update the docs
Date Tue, 21 May 2013 00:41:29 GMT
Updated Branches:
  refs/heads/master 96cc67417 -> 93a8770e4


http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/93a8770e/src/site/markdown/tutorial_spectator.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/tutorial_spectator.md b/src/site/markdown/tutorial_spectator.md
new file mode 100644
index 0000000..03fc45a
--- /dev/null
+++ b/src/site/markdown/tutorial_spectator.md
@@ -0,0 +1,78 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Helix Tutorial: Spectator
+
+Next, we\'ll learn how to implement a SPECTATOR.  Typically, a spectator needs to react to
changes within the distributed system.  Examples: a client that needs to know where to send
a request, a topic consumer in a consumer group.  The spectator is automatically informed
of changes in the _external state_ of the cluster, but it does not have to add any code to
keep track of other components in the system.
+
+### Start the Helix agent
+
+Same as for a PARTICIPANT, The Helix agent is the common component that connects each system
component with the controller.
+
+It requires the following parameters:
+
+* clusterName: A logical name to represent the group of nodes
+* instanceName: A logical name of the process creating the manager instance. Generally this
is host:port.
+* instanceType: Type of the process. This can be one of the following types, in this case,
use SPECTATOR:
+    * CONTROLLER: Process that controls the cluster, any number of controllers can be started
but only one will be active at any given time.
+    * PARTICIPANT: Process that performs the actual task in the distributed system.
+    * SPECTATOR: Process that observes the changes in the cluster.
+    * ADMIN: To carry out system admin actions.
+* zkConnectString: Connection string to Zookeeper. This is of the form host1:port1,host2:port2,host3:port3.
+
+After the Helix manager instance is created, only thing that needs to be registered is the
state model factory.
+The methods of the State Model will be called when controller sends transitions to the Participant.
 In this example, we'll use the OnlineOffline factory.  Other options:
+
+_NEED TO ADD OTHER OPTIONS HERE_
+
+### Spectator Code
+
+A spectator observes the cluster and is notified when the state of the system changes. Helix
consolidates the state of entire cluster in one Znode called ExternalView.
+Helix provides a default implementation RoutingTableProvider that caches the cluster state
and updates it when there is a change in the cluster.
+
+```
+manager = HelixManagerFactory.getZKHelixManager(clusterName,
+                                                          instanceName,
+                                                          InstanceType.PARTICIPANT,
+                                                          zkConnectString);
+manager.connect();
+RoutingTableProvider routingTableProvider = new RoutingTableProvider();
+manager.addExternalViewChangeListener(routingTableProvider);
+```
+
+In the following code snippet, the application sends the request to a valid instance by interrogating
the external view.  Suppose the desired resource for this request is in the partition myDB_1.
+
+```
+## instances = routingTableProvider.getInstances(, "PARTITION_NAME", "PARTITION_STATE");
+instances = routingTableProvider.getInstances("myDB", "myDB_1", "ONLINE");
+
+////////////////////////////////////////////////////////////////////////////////////////////////
+// Application-specific code to send a request to one of the instances                  
     //
+////////////////////////////////////////////////////////////////////////////////////////////////
+
+theInstance = instances.get(0);  // should choose an instance and throw an exception if none
are available
+result = theInstance.sendRequest(yourApplicationRequest, responseObject);
+
+```
+
+When the external view changes, the application needs to react by sending requests to a different
instance.  
+
+_SHOULD WE ADD AN EXAMPLE OF THIS?  IT WOULD MAKE THINGS MORE COMPLICATED, BECAUSE THE APP
SHOULD HAVE A LOCAL COPY OF THE ROUTING TABLE AND UPDATE IT ONLY WHEN THE EXTERNAL VIEW CHANGES,
THAT\'S HOW I'D IMPLEMENT IT._
+
+

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/93a8770e/src/site/markdown/tutorial_state.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/tutorial_state.md b/src/site/markdown/tutorial_state.md
new file mode 100644
index 0000000..cb51be9
--- /dev/null
+++ b/src/site/markdown/tutorial_state.md
@@ -0,0 +1,60 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Helix Tutorial: State Machine Configuration
+
+In this chapter, we\'ll learn about the state models provided by Helix, and how to create
your own custom state model.
+
+## State Models
+
+Helix comes with 3 default state models that are commonly used.  It is possible to have multiple
state models in a cluster. 
+Every resource that is added should be configured to use a state model that govern its _ideal
state_.
+
+### MASTER-SLAVE
+
+* Has 3 states: OFFLINE, SLAVE, MASTER
+* Maximum # of masters: 1
+* Slaves are based on the replication factor. Replication factor can be specified while adding
the resource
+
+
+### ONLINE-OFFLINE
+* Has 2 states: OFFLINE and ONLINE.  This simple state model is a good starting point for
most applications.
+
+### LEADER-STANDBY
+* 1 Leader and multiple stand-bys.  The idea is that exactly one leader accomplishes a designated
task, the stand-bys are ready to take over if the leader fails.
+
+## Constraints
+
+In addition to the state machine configuration, one can specify the constraints of states
and transitions.
+
+For example, one can say:
+* MASTER:1
+ Maximum number of replicas in MASTER state at any time is 1
+
+* OFFLINE-SLAVE:5 
+Maximum number of OFFLINE-SLAVE transitions that can happen concurrently in the system is
5 in this example.
+
+### State Priority
+
+Helix uses a greedy approach to satisfy the state constraints. For example, if the state
machine configuration says it needs 1 MASTER and 2 SLAVES, but only 1 node is active, Helix
must promote it to MASTER. This behavior is achieved by providing the state priority list
as MASTER,SLAVE.
+
+### State Transition Priority
+
+Helix tries to fire as many transitions as possible in parallel to reach the stable state
without violating constraints. By default, Helix simply sorts the transitions alphabetically
and fires as many as it can without violating the constraints. You can control this by overriding
the priority order.
+

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/93a8770e/src/site/markdown/tutorial_throttling.md
----------------------------------------------------------------------
diff --git a/src/site/markdown/tutorial_throttling.md b/src/site/markdown/tutorial_throttling.md
new file mode 100644
index 0000000..0fcc9bc
--- /dev/null
+++ b/src/site/markdown/tutorial_throttling.md
@@ -0,0 +1,35 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Helix Tutorial: Throttling
+
+In this chapter, we\'ll learn how to control the parallel execution of cluster tasks.  Only
a centralized cluster manager with global knowledge is capable of coordinating this decision.
+
+### Throttling
+
+Since all state changes in the system are triggered through transitions, Helix can control
the number of transitions that can happen in parallel. Some of the transitions may be light
weight, but some might involve moving data, which is quite expensive from a network and iops
perspective.
+
+Helix allows applications to set a threshold on transitions. The threshold can be set at
multiple scopes:
+
+* MessageType e.g STATE_TRANSITION
+* TransitionType e.g SLAVE-MASTER
+* Resource e.g database
+* Node i.e per-node maximum transitions in parallel
+
+_I can\'t find this code._  See HelixManager.getHelixAdmin.addMessageConstraint() 

http://git-wip-us.apache.org/repos/asf/incubator-helix/blob/93a8770e/src/site/site.xml
----------------------------------------------------------------------
diff --git a/src/site/site.xml b/src/site/site.xml
index 1bff519..e44a43e 100644
--- a/src/site/site.xml
+++ b/src/site/site.xml
@@ -58,11 +58,10 @@
 
     <menu name="Helix">
       <item name="Introduction" href="./index.html"/>
+      <item name="Core concepts" href="./Concepts.html"/>
+      <item name="Architecture" href="./Architecture.html"/>
       <item name="Quick Start" href="./Quickstart.html"/>
-      <item name="Core concept" href="./Concepts.html"/>
       <item name="Tutorial" href="./Tutorial.html"/>
-      <item name="Architecture" href="./Architecture.html"/>
-      <item name="Features" href="./Features.html"/>
       <item name="release ${currentRelease}" href="releasenotes/release-${currentRelease}.html"/>
       <item name="Download" href="./download.html"/>
     </menu>


Mime
View raw message