kafka-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From guozh...@apache.org
Subject kafka git commit: MINOR: Pass absolute directory path to RocksDB.open
Date Thu, 23 Jun 2016 21:36:42 GMT
Repository: kafka
Updated Branches:
  refs/heads/trunk 8ec4e4b7a -> 41a54358b


MINOR: Pass absolute directory path to RocksDB.open

The method `RocksDB.open` assumes an absolute file path. If a relative path is configured,
it leads to an exception like the following:
```
org.apache.kafka.streams.errors.ProcessorStateException: Error opening store CustomerIdToUserIdLookup
at location ./tmp/rocksdb/CustomerIdToUserIdLookup
	at org.rocksdb.RocksDB.open(Native Method)
	at org.rocksdb.RocksDB.open(RocksDB.java:183)
	at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:214)
	at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:165)
	at org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:170)
	at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:85)
	at org.apache.kafka.test.KStreamTestDriver.<init>(KStreamTestDriver.java:64)
	at org.apache.kafka.test.KStreamTestDriver.<init>(KStreamTestDriver.java:50)
	at com.simple.estuary.transform.streaming.CartesianTransactionEnrichmentJobTest.testBuilder(CartesianTransactionEnrichmentJobTest.java:41)
```

Is there any risk to always fetching the absolute path as proposed here?

Let me know if you think this requires a JIRA issue or a unit test. I started working on a
unit test, but don't know of a great solution for writing out a file to a relative directory.

This contribution is my original work and I license the work to the project under the project's
open source license.

Author: Jeff Klukas <jeff@klukas.net>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #1481 from jklukas/rocksdb-abspath


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/41a54358
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/41a54358
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/41a54358

Branch: refs/heads/trunk
Commit: 41a54358bececc42ba680267281ba01b3f33f77f
Parents: 8ec4e4b
Author: Jeff Klukas <jeff@klukas.net>
Authored: Thu Jun 23 14:36:38 2016 -0700
Committer: Guozhang Wang <wangguoz@gmail.com>
Committed: Thu Jun 23 14:36:38 2016 -0700

----------------------------------------------------------------------
 .../org/apache/kafka/streams/state/internals/RocksDBStore.java     | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/41a54358/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
----------------------------------------------------------------------
diff --git a/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
b/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
index d9670b3..4109ebf 100644
--- a/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
+++ b/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
@@ -212,7 +212,7 @@ public class RocksDBStore<K, V> implements KeyValueStore<K, V>
{
         try {
             if (ttl == TTL_NOT_USED) {
                 dir.getParentFile().mkdirs();
-                return RocksDB.open(options, dir.toString());
+                return RocksDB.open(options, dir.getAbsolutePath());
             } else {
                 throw new UnsupportedOperationException("Change log is not supported for
store " + this.name + " since it is TTL based.");
                 // TODO: support TTL with change log?


Mime
View raw message