cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jaakko Laine (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (CASSANDRA-785) Improve load balancing when using rack aware or DC strategy
Date Tue, 06 Apr 2010 18:01:34 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854086#action_12854086
] 

Jaakko Laine edited comment on CASSANDRA-785 at 4/6/10 5:59 PM:
----------------------------------------------------------------

As discussed earlier in mailing list, load balancing cannot simply consider primary range
of bootstrap source when using rack aware (or DC shard) strategy, but whole range (primary
+ replicas) need to be taken into account. Following is a result of one test run using current
LB functionality:

192.168.0.104  100.84 MB     573371861609767657238978844007740582
192.168.0.101  152.79 MB     37375360823777355034255131821084484487
192.168.0.102  171.61 MB     78679861790617677332793962569191145101
192.178.0.109  688.97 MB     81901584797347486028896754414340634003
192.178.0.108  21.22 MB      83864326228946960659296251414309936872
192.168.1.107  598.99 MB     89819165849860700111165509269175012567
192.168.1.106  40.53 MB      98071312689481731589290075340406457109
192.168.1.105  65.89 MB      114171242327420423416372328565954705130
192.168.0.103  250.74 MB     146240099105929890127228403396377438645

As can be seen, the load is not exactly evenly balanced.

Attached patch gets bootstrap token according to the whole range of the bootstrap source.
Additionally, the token is adjusted locally in order to avoid nodes in different racks from
bootstrapping too close to each other. Almost similar test run was done after the modifications
and the final situation was:

192.168.1.105  187.56 MB     6397937533436544563999349762491808989
192.178.0.108  195.59 MB     22595489350255970966018220707106798316
192.168.0.101  187.18 MB     37375360823777355034255131821084484487
192.168.0.102  241.7 MB      78679861790617677332793962569191145101
192.168.1.106  305.2 MB      98034334071769068450923602160301798881
192.178.0.109  343.16 MB     107128436236301245608915524607283274430
192.168.0.104  142.75 MB     114572714021825029937699759732741779443
192.168.1.107  134.35 MB     131133602853429031188902390794550667876
192.168.0.103  129.2 MB      146240099105929890127228403396377438645
192.178.0.110  224.89 MB     162707623323903830430688125027655118960

In this case nodes in different racks end up distributed much better in the ring.

Edit: token rings were messy, cleaned up a bit

      was (Author: jaakko):
    As discussed earlier in mailing list, load balancing cannot simply consider primary range
of bootstrap source when using rack aware (or DC shard) strategy, but whole range (primary
+ replicas) need to be taken into account. Following is a result of one test run using current
LB functionality:

Address       Status     Load          Range                                      Ring
                                       146240099105929890127228403396377438645    
192.168.0.104 Up         100.84 MB     573371861609767657238978844007740582       |<--|
192.168.0.101 Up         152.79 MB     37375360823777355034255131821084484487     |   ^
192.168.0.102 Up         171.61 MB     78679861790617677332793962569191145101     v   |
192.178.0.109 Up         688.97 MB     81901584797347486028896754414340634003     |   ^
192.178.0.108 Up         21.22 MB      83864326228946960659296251414309936872     v   |
192.168.1.107 Up         598.99 MB     89819165849860700111165509269175012567     |   ^
192.168.1.106 Up         40.53 MB      98071312689481731589290075340406457109     v   |
192.168.1.105 Up         65.89 MB      114171242327420423416372328565954705130    |   ^
192.168.0.103 Up         250.74 MB     146240099105929890127228403396377438645    |-->|

As can be seen, the load is not exactly evenly balanced.

Attached patch gets bootstrap token according to the whole range of the bootstrap source.
Additionally, the token is adjusted locally in order to avoid nodes in different racks from
bootstrapping too close to each other. Almost similar test run was done after the modifications
and the final situation was:

Address       Status     Load          Range                                      Ring
                                       162707623323903830430688125027655118960    
192.168.1.105 Up         187.56 MB     6397937533436544563999349762491808989      |<--|
192.178.0.108 Up         195.59 MB     22595489350255970966018220707106798316     |   ^
192.168.0.101 Up         187.18 MB     37375360823777355034255131821084484487     v   |
192.168.0.102 Up         241.7 MB      78679861790617677332793962569191145101     |   ^
192.168.1.106 Up         305.2 MB      98034334071769068450923602160301798881     v   |
192.178.0.109 Up         343.16 MB     107128436236301245608915524607283274430    |   ^
192.168.0.104 Up         142.75 MB     114572714021825029937699759732741779443    v   |
192.168.1.107 Up         134.35 MB     131133602853429031188902390794550667876    |   ^
192.168.0.103 Up         129.2 MB      146240099105929890127228403396377438645    v   |
192.178.0.110 Up         224.89 MB     162707623323903830430688125027655118960    |-->|

In this case nodes in different racks end up distributed much better in the ring.

  
> Improve load balancing when using rack aware or DC strategy
> -----------------------------------------------------------
>
>                 Key: CASSANDRA-785
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-785
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.5
>            Reporter: Jaakko Laine
>            Assignee: Jaakko Laine
>             Fix For: 0.8
>
>         Attachments: 785.patch
>
>
> Current load balancing functionality does not consider data centers. This may result
in new nodes bootstrapping in "wrong" place if most loaded node is in another DC than the
bootstrapping node. Explore possibilities to make load balance work better in multi-DC configuration
without making load balancing complicated.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message