cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jbel...@apache.org
Subject [1/3] git commit: Scale hinted_handoff_throttle_in_kb to cluster size patch by jbellis; reviewed by rbranson for CASSANDRA-5272
Date Wed, 29 May 2013 20:00:15 GMT
Updated Branches:
  refs/heads/cassandra-1.2 87450cb13 -> 19d2782c7
  refs/heads/trunk 4f7e2b801 -> f3b1a2cbc


Scale hinted_handoff_throttle_in_kb to cluster size
patch by jbellis; reviewed by rbranson for CASSANDRA-5272


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19d2782c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19d2782c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19d2782c

Branch: refs/heads/cassandra-1.2
Commit: 19d2782c715a0d0f73958edce3f4be3709f92d18
Parents: 87450cb
Author: Jonathan Ellis <jbellis@apache.org>
Authored: Wed May 29 14:59:45 2013 -0500
Committer: Jonathan Ellis <jbellis@apache.org>
Committed: Wed May 29 14:59:45 2013 -0500

----------------------------------------------------------------------
 CHANGES.txt                                        |    1 +
 NEWS.txt                                           |   10 ++++++++++
 conf/cassandra.yaml                                |    6 +++++-
 .../apache/cassandra/db/HintedHandOffManager.java  |    4 +++-
 4 files changed, 19 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cassandra/blob/19d2782c/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index f51baae..2b8e01f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 1.2.6
+ * Scale hinted_handoff_throttle_in_kb to cluster size (CASSANDRA-5272)
  * (Hadoop) Fix InputKeyRange in CFIF (CASSANDRA-5536)
  * Fix dealing with ridiculously large max sstable sizes in LCS (CASSANDRA-5589)
  * Ignore pre-truncate hints (CASSANDRA-4655)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/19d2782c/NEWS.txt
----------------------------------------------------------------------
diff --git a/NEWS.txt b/NEWS.txt
index 706cd29..099e366 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -8,6 +8,16 @@ upgrade, just in case you need to roll back to the previous version.
 (Cassandra version X + 1 will always be able to read data files created
 by version X, but the inverse is not necessarily the case.)
 
+1.2.6
+=====
+
+Upgrading
+---------
+    - hinted_handoff_throttle_in_kb is now reduced by a factor
+      proportional to the number of nodes in the cluster (see
+      https://issues.apache.org/jira/browse/CASSANDRA-5272).
+
+
 1.2.5
 =====
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/19d2782c/conf/cassandra.yaml
----------------------------------------------------------------------
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 4d2eff3..2d2b484 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -42,7 +42,11 @@ hinted_handoff_enabled: true
 # generated.  After it has been dead this long, new hints for it will not be
 # created until it has been seen alive and gone down again.
 max_hint_window_in_ms: 10800000 # 3 hours
-# throttle in KBs per second, per delivery thread
+# Maximum throttle in KBs per second, per delivery thread.  This will be
+# reduced proportionally to the number of nodes in the cluster.  (If there
+# are two nodes in the cluster, each delivery thread will use the maximum
+# rate; if there are three, each will throttle to half of the maximum,
+# since we expect two nodes to be delivering hints simultaneously.)
 hinted_handoff_throttle_in_kb: 1024
 # Number of threads with which to deliver hints;
 # Consider increasing this number when you have multi-dc deployments, since

http://git-wip-us.apache.org/repos/asf/cassandra/blob/19d2782c/src/java/org/apache/cassandra/db/HintedHandOffManager.java
----------------------------------------------------------------------
diff --git a/src/java/org/apache/cassandra/db/HintedHandOffManager.java b/src/java/org/apache/cassandra/db/HintedHandOffManager.java
index 9346fb3..5ca32c3 100644
--- a/src/java/org/apache/cassandra/db/HintedHandOffManager.java
+++ b/src/java/org/apache/cassandra/db/HintedHandOffManager.java
@@ -300,7 +300,9 @@ public class HintedHandOffManager implements HintedHandOffManagerMBean
         logger.debug("Using pageSize of {}", pageSize);
 
         // rate limit is in bytes per second. Uses Double.MAX_VALUE if disabled (set to 0
in cassandra.yaml).
-        int throttleInKB = DatabaseDescriptor.getHintedHandoffThrottleInKB();
+        // max rate is scaled by the number of nodes in the cluster (CASSANDRA-5272).
+        int throttleInKB = DatabaseDescriptor.getHintedHandoffThrottleInKB()
+                           / (StorageService.instance.getTokenMetadata().getAllEndpoints().size()
- 1);
         RateLimiter rateLimiter = RateLimiter.create(throttleInKB == 0 ? Double.MAX_VALUE
: throttleInKB * 1024);
 
         delivery:


Mime
View raw message