hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Feng Honghua (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-9501) No throttling for replication
Date Wed, 11 Sep 2013 10:00:52 GMT
Feng Honghua created HBASE-9501:

             Summary: No throttling for replication
                 Key: HBASE-9501
                 URL: https://issues.apache.org/jira/browse/HBASE-9501
             Project: HBase
          Issue Type: Improvement
          Components: Replication
            Reporter: Feng Honghua

When we disable a peer for a time of period, and then enable it, the ReplicationSource in
master cluster will push the accumulated hlog entries during the disabled interval to the
re-enabled peer cluster at full speed.

If the bandwidth of the two clusters is shared by different applications, the push at full
speed for replication can use all the bandwidth and severely influence other applications.

Though there are two config replication.source.size.capacity and replication.source.nb.capacity
to tweak the batch size each time a push delivers, but if decrease these two configs, the
number of pushes increase, and all these pushes proceed continuously without pause. And no
obvious help for the bandwidth throttling.

>From bandwidth-sharing and push-speed perspective, it's more reasonable to provide a bandwidth
up limit for each peer push channel, and within that limit, peer can choose a big batch size
for each push for bandwidth efficiency.

Any opinion?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message