helix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jiajun Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HELIX-659) Extend Helix to Support Resource with Multiple States
Date Mon, 10 Jul 2017 19:09:00 GMT

     [ https://issues.apache.org/jira/browse/HELIX-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jiajun Wang updated HELIX-659:
------------------------------
    Summary: Extend Helix to Support Resource with Multiple States  (was: Support Additional
Associate States)

> Extend Helix to Support Resource with Multiple States
> -----------------------------------------------------
>
>                 Key: HELIX-659
>                 URL: https://issues.apache.org/jira/browse/HELIX-659
>             Project: Apache Helix
>          Issue Type: New Feature
>          Components: helix-core
>    Affects Versions: 0.6.x
>            Reporter: Jiajun Wang
>
> Currently, Helix only supports management a single state for all resources/partitions.
However, in the real world, cluster management requirements may be more complicated than that.
> In Pinot, for example, each partition need to be assigned a version for ensuring data
consistency.
> When a new version comes, the system needs to replace the old partition with the new
one. And the replacement is done one partition by one partition. So any reads during this
period will get inconsistent data.
> Pinot system cannot directly put the version information into the section(partition)
state field because it is already occupied by the main state (offline-online for instance)
used by Helix controller.
> So Pinot team relies on some workarounds to implement their application logic: creating
a new resource with the latest version and replace them after the resource is fully loaded.
And for Helix controller, version is unknown.
> Another option is Pinot team maintaining their own config item or property store item
for recording versions.
> Both ways require Pinot team implementing version control themselves.
> Another requirement is from Ambry team. Where partition can be "ONLINE:READ" or "ONLINE:WRITE".
> In both cases, single state mechanism is not sufficient for applications' requirement.
> It would be very helpful to provide a framework level feature that supports more than
one states for each partition.
> Benefits: 
> # The application doesn't need to write additional code for managing additional states.
> # Avoid potential conflict when multiple states transition happens concurrently.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message