kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From guozh...@apache.org
Subject kafka git commit: MINOR: increase RocksDb parallelism
Date Fri, 10 Mar 2017 21:43:07 GMT
Repository: kafka
Updated Branches:
  refs/heads/trunk dfa2ef483 -> 0011a3dc3


MINOR: increase RocksDb parallelism

Author: Eno Thereska <eno@confluent.io>
Author: Eno Thereska <eno.thereska@gmail.com>

Reviewers: Damian Guy, Guozhang Wang

Closes #2663 from enothereska/minor-rocksdb-parallel


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/0011a3dc
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/0011a3dc
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/0011a3dc

Branch: refs/heads/trunk
Commit: 0011a3dc337a41ceb21b1567db7bdc3219a07552
Parents: dfa2ef4
Author: Eno Thereska <eno@confluent.io>
Authored: Fri Mar 10 13:43:04 2017 -0800
Committer: Guozhang Wang <wangguoz@gmail.com>
Committed: Fri Mar 10 13:43:04 2017 -0800

----------------------------------------------------------------------
 .../org/apache/kafka/streams/state/internals/RocksDBStore.java   | 4 ++++
 1 file changed, 4 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/0011a3dc/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
----------------------------------------------------------------------
diff --git a/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
b/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
index 823ad47..932ddd2 100644
--- a/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
+++ b/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
@@ -124,6 +124,10 @@ public class RocksDBStore<K, V> implements KeyValueStore<K,
V> {
         options.setCreateIfMissing(true);
         options.setErrorIfExists(false);
         options.setInfoLogLevel(InfoLogLevel.ERROR_LEVEL);
+        // this is the recommended way to increase parallelism in RocksDb
+        // note that the current implementation increases the number of compaction threads
+        // but not flush threads.
+        options.setIncreaseParallelism(Runtime.getRuntime().availableProcessors());
 
         wOptions = new WriteOptions();
         wOptions.setDisableWAL(true);


Mime
View raw message