Mailing-List: contact commits-help@helix.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@helix.apache.org
Date: Mon, 10 Jul 2017 19:11:00 +0000 (UTC)
From: "Jiajun Wang (JIRA)" <jira@apache.org>
To: commits@helix.incubator.apache.org
Message-ID: <JIRA.13076107.1496214571000.210309.1499713860082@Atlassian.JIRA>
In-Reply-To: <JIRA.13076107.1496214571000@Atlassian.JIRA>
References: <JIRA.13076107.1496214571000@Atlassian.JIRA> <JIRA.13076107.1496214571143@jira-lw-us.apache.org>
Subject: [jira] [Updated] (HELIX-659) Extend Helix to Support Resource with
 Multiple States
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 10 Jul 2017 19:11:11 -0000


     [ https://issues.apache.org/jira/browse/HELIX-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jiajun Wang updated HELIX-659:
------------------------------
    Description: 
h1. Problem Statement

h2. Single State Model v.s. Multiple State Models

Currently, Each Helix resource is associated with a single state model, and each replica of a partition can only be in any one of these states defined in the state model at any time. And Helix manages state transition based on the single state model.

!https://documents.lucidchart.com/documents/e19ab04e-aa06-4ab3-9e57-cfe273554fa1/pages/0_0?a=2416&x=-11&y=71&w=517&h=198&store=1&accept=image%2F*&auth=LCA%20313ced8fb855e8fc1a7043f7fe91cdfa15fffb6b-ts%3D1498857664!

However, in many scenarios, resources could be more complicated to be modeled by a single state model.
As an example, partitions from a resource could be described in different dimensions: SlaveMaster state, Read or Write state and its versions. They represent different dimensions of the overall resource status. States from each dimension are based on different state models. Note that we have state machines simplified in this document.

!https://documents.lucidchart.com/documents/e19ab04e-aa06-4ab3-9e57-cfe273554fa1/pages/0_0?a=2416&x=-71&y=66&w=1822&h=308&store=1&accept=image%2F*&auth=LCA%2041fa743ba130f41786dee3527de6206cebdd4534-ts%3D1498857664!

The basic idea is that states in these 3 dimensions are in parallel and can be changed independently. For instance, R/W state may be changed without updating slave/master state.

h2. Finite State Machine v.s. Dynamic State Model

In addition, Helix employs finite state machine to define a state model. However, some state model can not be easily modeled by a finite state machine with fixed states, for example, the versions.  We call such state model as the dynamic state model. It is read, set, and understood by the application. We will need to extend Helix to support such dynamic state model. Note that Helix should not and will not be able to calculate the best possible dynamic states.

The version of a software is one of the best examples to understand dynamic state.

Let's consider one application that is deployed on multiple nodes, which work together as a cluster. The green node works as the master, and all dark blue nodes are slaves. When Admins upgrades the service from 1.0.0 to 1.1.0, they need to ensure upgrading all nodes to the new version and then claim upgrade is done. After the upgrade process, it is important to ensure that all software versions are consistent.

If Helix framework is leveraged to support upgrading the cluster, it will help to simplify application logic and ensure consistency. For instance, the service (cluster) itself is regarded as the resource. And each node is mapped as a partition. Then upgrading is simply a state transition. Admins can check external view for ensuring consistency.
Note that during this version upgrade, the master node is still master node, and slave nodes are still slave nodes. So the version state is parallel to the other states.

  was:
h1. Problem Statement

h2. Single State Model v.s. Multiple State Models

Currently, Each Helix resource is associated with a single state model, and each replica of a partition can only be in any one of these states defined in the state model at any time. And Helix manages state transition based on the single state model.

https://documents.lucidchart.com/documents/e19ab04e-aa06-4ab3-9e57-cfe273554fa1/pages/0_0?a=2416&x=-11&y=71&w=517&h=198&store=1&accept=image%2F*&auth=LCA%20313ced8fb855e8fc1a7043f7fe91cdfa15fffb6b-ts%3D1498857664

However, in many scenarios, resources could be more complicated to be modeled by a single state model.
As an example, partitions from a resource could be described in different dimensions: SlaveMaster state, Read or Write state and its versions. They represent different dimensions of the overall resource status. States from each dimension are based on different state models. Note that we have state machines simplified in this document.

https://documents.lucidchart.com/documents/e19ab04e-aa06-4ab3-9e57-cfe273554fa1/pages/0_0?a=2416&x=-71&y=66&w=1822&h=308&store=1&accept=image%2F*&auth=LCA%2041fa743ba130f41786dee3527de6206cebdd4534-ts%3D1498857664

The basic idea is that states in these 3 dimensions are in parallel and can be changed independently. For instance, R/W state may be changed without updating slave/master state.

h2. Finite State Machine v.s. Dynamic State Model

In addition, Helix employs finite state machine to define a state model. However, some state model can not be easily modeled by a finite state machine with fixed states, for example, the versions.  We call such state model as the dynamic state model. It is read, set, and understood by the application. We will need to extend Helix to support such dynamic state model. Note that Helix should not and will not be able to calculate the best possible dynamic states.

The version of a software is one of the best examples to understand dynamic state.

Let's consider one application that is deployed on multiple nodes, which work together as a cluster. The green node works as the master, and all dark blue nodes are slaves. When Admins upgrades the service from 1.0.0 to 1.1.0, they need to ensure upgrading all nodes to the new version and then claim upgrade is done. After the upgrade process, it is important to ensure that all software versions are consistent.

If Helix framework is leveraged to support upgrading the cluster, it will help to simplify application logic and ensure consistency. For instance, the service (cluster) itself is regarded as the resource. And each node is mapped as a partition. Then upgrading is simply a state transition. Admins can check external view for ensuring consistency.
Note that during this version upgrade, the master node is still master node, and slave nodes are still slave nodes. So the version state is parallel to the other states.


> Extend Helix to Support Resource with Multiple States
> -----------------------------------------------------
>
>                 Key: HELIX-659
>                 URL: https://issues.apache.org/jira/browse/HELIX-659
>             Project: Apache Helix
>          Issue Type: New Feature
>          Components: helix-core
>    Affects Versions: 0.6.x
>            Reporter: Jiajun Wang
>
> h1. Problem Statement
> h2. Single State Model v.s. Multiple State Models
> Currently, Each Helix resource is associated with a single state model, and each replica of a partition can only be in any one of these states defined in the state model at any time. And Helix manages state transition based on the single state model.
> !https://documents.lucidchart.com/documents/e19ab04e-aa06-4ab3-9e57-cfe273554fa1/pages/0_0?a=2416&x=-11&y=71&w=517&h=198&store=1&accept=image%2F*&auth=LCA%20313ced8fb855e8fc1a7043f7fe91cdfa15fffb6b-ts%3D1498857664!
> However, in many scenarios, resources could be more complicated to be modeled by a single state model.
> As an example, partitions from a resource could be described in different dimensions: SlaveMaster state, Read or Write state and its versions. They represent different dimensions of the overall resource status. States from each dimension are based on different state models. Note that we have state machines simplified in this document.
> !https://documents.lucidchart.com/documents/e19ab04e-aa06-4ab3-9e57-cfe273554fa1/pages/0_0?a=2416&x=-71&y=66&w=1822&h=308&store=1&accept=image%2F*&auth=LCA%2041fa743ba130f41786dee3527de6206cebdd4534-ts%3D1498857664!
> The basic idea is that states in these 3 dimensions are in parallel and can be changed independently. For instance, R/W state may be changed without updating slave/master state.
> h2. Finite State Machine v.s. Dynamic State Model
> In addition, Helix employs finite state machine to define a state model. However, some state model can not be easily modeled by a finite state machine with fixed states, for example, the versions.  We call such state model as the dynamic state model. It is read, set, and understood by the application. We will need to extend Helix to support such dynamic state model. Note that Helix should not and will not be able to calculate the best possible dynamic states.
> The version of a software is one of the best examples to understand dynamic state.
> Let's consider one application that is deployed on multiple nodes, which work together as a cluster. The green node works as the master, and all dark blue nodes are slaves. When Admins upgrades the service from 1.0.0 to 1.1.0, they need to ensure upgrading all nodes to the new version and then claim upgrade is done. After the upgrade process, it is important to ensure that all software versions are consistent.
> If Helix framework is leveraged to support upgrading the cluster, it will help to simplify application logic and ensure consistency. For instance, the service (cluster) itself is regarded as the resource. And each node is mapped as a partition. Then upgrading is simply a state transition. Admins can check external view for ensuring consistency.
> Note that during this version upgrade, the master node is still master node, and slave nodes are still slave nodes. So the version state is parallel to the other states.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)