helix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HELIX-599) Support creating/maintaining/routing resources with same names in different instance groups
Date Sun, 05 Jul 2015 06:32:04 GMT

    [ https://issues.apache.org/jira/browse/HELIX-599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14614163#comment-14614163
] 

ASF GitHub Bot commented on HELIX-599:
--------------------------------------

Github user kanakb commented on a diff in the pull request:

    https://github.com/apache/helix/pull/31#discussion_r33892312
  
    --- Diff: helix-core/src/main/java/org/apache/helix/spectator/RoutingTableProvider.java
---
    @@ -235,8 +376,94 @@ PartitionInfo get(String stateUnitKey) {
         }
       }
     
    +  class ResourceGroupInfo {
    +    // store PartitionInfo for each partition for all resources in the resource group
    +    HashMap<String, PartitionInfo> partitionInfoMap;
    +    // stores the Set of Instances in a given state for all resources in the resource
group
    +    HashMap<String, Set<InstanceConfig>> stateInfoMap;
    +    // stores the ResourceInfo for each resource with different instance tag.
    +    HashMap<String, ResourceInfo> tagResourceInfoMap;
    +
    +    public ResourceGroupInfo() {
    +      partitionInfoMap = new HashMap<String, RoutingTableProvider.PartitionInfo>();
    +      stateInfoMap = new HashMap<String, Set<InstanceConfig>>();
    +      tagResourceInfoMap = new HashMap<String, ResourceInfo>();
    +    }
    +
    +    public void addEntry(String resourceTag, String stateUnitKey, String state, InstanceConfig
config) {
    +      // add
    +      if (!stateInfoMap.containsKey(state)) {
    +        Comparator<InstanceConfig> comparator = new Comparator<InstanceConfig>()
{
    +
    +          @Override
    +          public int compare(InstanceConfig o1, InstanceConfig o2) {
    +            if (o1 == o2) {
    +              return 0;
    +            }
    +            if (o1 == null) {
    +              return -1;
    +            }
    +            if (o2 == null) {
    +              return 1;
    +            }
    +
    +            int compareTo = o1.getHostName().compareTo(o2.getHostName());
    +            if (compareTo == 0) {
    +              return o1.getPort().compareTo(o2.getPort());
    +            } else {
    +              return compareTo;
    +            }
    +
    +          }
    +        };
    +        stateInfoMap.put(state, new TreeSet<InstanceConfig>(comparator));
    +      }
    +      Set<InstanceConfig> set = stateInfoMap.get(state);
    +      set.add(config);
    +
    +      if (!partitionInfoMap.containsKey(stateUnitKey)) {
    +        partitionInfoMap.put(stateUnitKey, new PartitionInfo());
    +      }
    +      PartitionInfo stateUnitKeyInfo = partitionInfoMap.get(stateUnitKey);
    +      stateUnitKeyInfo.addEntry(state, config);
    +
    +      if (!tagResourceInfoMap.containsKey(resourceTag)) {
    +        tagResourceInfoMap.put(resourceTag, new ResourceInfo());
    +      }
    +      ResourceInfo resourceInfo = tagResourceInfoMap.get(resourceTag);
    +      resourceInfo.addEntry(stateUnitKey, state, config);
    +    }
    +
    +    public Set<InstanceConfig> getInstances(String state) {
    +      Set<InstanceConfig> instanceSet = stateInfoMap.get(state);
    +      return instanceSet;
    +    }
    +
    +    public Set<InstanceConfig> getInstances(String state, String resourceTag) {
    +      ResourceInfo resourceInfo = tagResourceInfoMap.get(resourceTag);
    --- End diff --
    
    What if multiple resources have the same tag?


> Support creating/maintaining/routing resources with same names in different instance
groups
> -------------------------------------------------------------------------------------------
>
>                 Key: HELIX-599
>                 URL: https://issues.apache.org/jira/browse/HELIX-599
>             Project: Apache Helix
>          Issue Type: New Feature
>          Components: helix-core, helix-webapp-admin
>            Reporter: Lei Xia
>            Assignee: Lei Xia
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> In LinkedIn, we have a new use scenario that there will be multiple databases sitting
in the same Helix cluster with the same name, but on different instance groups.  What we need
are:
>  1) Allow resources (databases) with the same name, these resources are on different
instance groups (with different tags).
>  2) Routing table (Spectator) is able to aggregate and return all instance (from multiple
instance groups) that hold the database with given name.
> Our proposed solution is:
>  1) Add a "Resource Group" field in IdealState for the databases with the same names
from different instance groups
>  2) Use Instance Group Tag (or new "Resource Tag") to differentiate databases (with same
name) from different instance groups.
>  3) Use name mangling for Idealstate, for example, with database TestDB in instance group
"testGroup", the IdealState and ExternalView id would be "TestDB$testGroup". 
>  4) Change Helix Routing Table to be able to aggregate databases from the same resource
group.
>  
> Four new APIs are going to be added to RoutingTableProvider:
> public class RoutingTableProvider {
>  
> /**
>  * returns the instances that contain the given partition in a specific state from all
resources with given resource name
>  */
> public List<InstanceConfig> getInstances(String resource, String partition, String
state);
>  
> /**
>  * returns the instances that contain the given partition in a specific state from selected
resources with given name and tags
>  */
> public List<InstanceConfig> getInstances(String resource, String partition, String
state, List<String> resourceTags);
>  
> /**
>  * returns instances that contain given resource that are in a specific state
>  */
> public Set<InstanceConfig> getInstances(String resource, String state);
>  
> /**
>  * returns instances that contain given resource with tags that are in a specific state
>  */
> public Set<InstanceConfig> getInstances(String resource, String state,  List<String>
groupTags);
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message