helix-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HELIX-635) GenericTaskAssignmentCalculator does not balance task assignment
Date Tue, 13 Sep 2016 23:06:22 GMT

    [ https://issues.apache.org/jira/browse/HELIX-635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488751#comment-15488751
] 

ASF GitHub Bot commented on HELIX-635:
--------------------------------------

GitHub user dasahcc opened a pull request:

    https://github.com/apache/helix/pull/51

    [HELIX-635] GenericTaskAssignmentCalculator does not balance task assignment

    Currently, GenericTaskAssignmentCalculator using AutoRebalancerStrategy compute the assignment
mapping. There is a bug that all the tasks from same job will be assigned to one node, which
is not balanced.
    
    This fix contains:
        1. Implement consistent hashing mapping calculation for GenericTaskAssignmentCalculator
        2. Remove reassign logics and applied in consistent hashing
        3. Add tests for GenericTaskAssignmentCalculator
        4. Refactoring TaskAssignmentCalculator API, since ClusterDataCache is too large and
not all the contents inside are used.
        5. Support identical task initialization with job command and number of tasks
        6. Remove unused MaxForcedReassignmentPerTask field
        7. Refactor logics of failure.
        

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dasahcc/helix helix-0.6.x

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/helix/pull/51.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #51
    
----
commit 0a18726fcad7b8a0fe5e77d7a2c9848b86461ccc
Author: Junkai Xue <jxue@linkedin.com>
Date:   2016-09-13T22:28:57Z

    [HELIX-635] GenericTaskAssignmentCalculator rebalance with consistent hashing
    
    1. Implement consistent hashing mapping calculation
    2. Remove reassign logics and applied in consistent hashing
    3. Add tests for GenericTaskAssignmentCalculator

commit 7bb2a9db2396a00bb9a721634a2432240679c657
Author: Junkai Xue <jxue@linkedin.com>
Date:   2016-09-13T23:00:08Z

    Refactor TaskAssignmentCalculator API
    
    Refactoring TaskAssignmentCalculator API, since ClusterDataCache is too large and not
all the contents inside are used.

commit 27cb5ecfe5f3b07c5c97534222558702a05e5aae
Author: Junkai Xue <jxue@linkedin.com>
Date:   2016-09-13T23:01:39Z

    Job Config and logic refactoring
    
    1. Support identical task initialization with job command and number of tasks
    2. Remove unused MaxForcedReassignmentPerTask field
    3. Refactor logics of failure.

----


> GenericTaskAssignmentCalculator does not balance task assignment
> ----------------------------------------------------------------
>
>                 Key: HELIX-635
>                 URL: https://issues.apache.org/jira/browse/HELIX-635
>             Project: Apache Helix
>          Issue Type: Bug
>          Components: helix-core
>            Reporter: Junkai Xue
>            Assignee: Junkai Xue
>             Fix For: 0.6.x
>
>
> Currently, GenericTaskAssignmentCalculator using AutoRebalancerStrategy compute the assignment
mapping. There is a bug that all the tasks from same job will be assigned to one node, which
is not balanced.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message