helix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HELIX-599) Support creating/maintaining/routing resources with same names in different instance groups
Date Fri, 10 Jul 2015 05:23:04 GMT

    [ https://issues.apache.org/jira/browse/HELIX-599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621752#comment-14621752
] 

ASF GitHub Bot commented on HELIX-599:
--------------------------------------

Github user lei-xia commented on a diff in the pull request:

    https://github.com/apache/helix/pull/31#discussion_r34328288
  
    --- Diff: helix-core/src/main/java/org/apache/helix/model/IdealState.java ---
    @@ -55,7 +55,9 @@
         MAX_PARTITIONS_PER_INSTANCE,
         INSTANCE_GROUP_TAG,
         REBALANCER_CLASS_NAME,
    -    HELIX_ENABLED
    +    HELIX_ENABLED,
    +    RESOURCE_GROUP_NAME,
    +    RESOURCE_GROUP_ENABLED
    --- End diff --
    
    ResourceGroupEnabled may be a little confusing, I will change it to GroupRoutingEnabled.
This means routingProvider should consider aggregate all partition/instances from all resources
from the same resourceGroup.  ResourceGroup Name and Instance Tag for a resource can be used
for other purpose, only when the GroupRoutingEnabled is true, they are combined together to
provide this group routing feature.  This also says, GroupRoutingEnabled is true implicitly
means ResourceGroup and ResourceTag (instanceTag) are set in the IdealState, and resourceTag
is identical in each resource from the same resource group.   But this constraints are not
necessary if GroupRoutingEnable is false.  For example, now many resources could have same
instanceGroupTag. 


> Support creating/maintaining/routing resources with same names in different instance
groups
> -------------------------------------------------------------------------------------------
>
>                 Key: HELIX-599
>                 URL: https://issues.apache.org/jira/browse/HELIX-599
>             Project: Apache Helix
>          Issue Type: New Feature
>          Components: helix-core, helix-webapp-admin
>            Reporter: Lei Xia
>            Assignee: Lei Xia
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> In LinkedIn, we have a new use scenario that there will be multiple databases sitting
in the same Helix cluster with the same name, but on different instance groups.  What we need
are:
>  1) Allow resources (databases) with the same name, these resources are on different
instance groups (with different tags).
>  2) Routing table (Spectator) is able to aggregate and return all instance (from multiple
instance groups) that hold the database with given name.
> Our proposed solution is:
>  1) Add a "Resource Group" field in IdealState for the databases with the same names
from different instance groups
>  2) Use Instance Group Tag (or new "Resource Tag") to differentiate databases (with same
name) from different instance groups.
>  3) Use name mangling for Idealstate, for example, with database TestDB in instance group
"testGroup", the IdealState and ExternalView id would be "TestDB$testGroup". 
>  4) Change Helix Routing Table to be able to aggregate databases from the same resource
group.
>  
> Four new APIs are going to be added to RoutingTableProvider:
> public class RoutingTableProvider {
>  
> /**
>  * returns the instances that contain the given partition in a specific state from all
resources with given resource name
>  */
> public List<InstanceConfig> getInstances(String resource, String partition, String
state);
>  
> /**
>  * returns the instances that contain the given partition in a specific state from selected
resources with given name and tags
>  */
> public List<InstanceConfig> getInstances(String resource, String partition, String
state, List<String> resourceTags);
>  
> /**
>  * returns instances that contain given resource that are in a specific state
>  */
> public Set<InstanceConfig> getInstances(String resource, String state);
>  
> /**
>  * returns instances that contain given resource with tags that are in a specific state
>  */
> public Set<InstanceConfig> getInstances(String resource, String state,  List<String>
groupTags);
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message