Mailing-List: contact jira-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@kafka.apache.org
Date: Wed, 8 Nov 2017 11:11:00 +0000 (UTC)
From: "James Wu (JIRA)" <jira@apache.org>
To: jira@kafka.apache.org
Message-ID: <JIRA.13116944.1510139415000.180456.1510139460086@Atlassian.JIRA>
In-Reply-To: <JIRA.13116944.1510139415000@Atlassian.JIRA>
References: <JIRA.13116944.1510139415000@Atlassian.JIRA> <JIRA.13116944.1510139415824@jira-lw-us.apache.org>
Subject: [jira] [Created] (KAFKA-6186) RocksDB based WindowStore fail to
 create db file on Windows OS
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 08 Nov 2017 11:11:04 -0000

James Wu created KAFKA-6186:
-------------------------------

             Summary: RocksDB based WindowStore fail to create db file on Windows OS
                 Key: KAFKA-6186
                 URL: https://issues.apache.org/jira/browse/KAFKA-6186
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 1.0.0
         Environment: Windows OS
            Reporter: James Wu


Code snippet just like below

...
        textLines.flatMapValues(value -> Arrays.asList(pattern.split(value.toLowerCase()))).groupBy((key, word) -> word)
                .windowedBy(TimeWindows.of(10000)).count(Materialized.as("Counts"));
...

Run it on Windows, then the exception is throw as below

Caused by: org.rocksdb.RocksDBException: Failed to create dir: F:\tmp\kafka-streams\wordcount-lambda-example\1_0\Counts\Counts:1510099200000: Invalid argument
	at org.rocksdb.RocksDB.open(Native Method) ~[rocksdbjni-5.7.3.jar:na]
	at org.rocksdb.RocksDB.open(RocksDB.java:231) ~[rocksdbjni-5.7.3.jar:na]
	at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:197) ~[kafka-streams-1.0.0.jar:na]
	... 29 common frames omitted

Checked the code, I found the issue is caused by line 72 in org.apache.kafka.streams.state.internals.Segments

    String segmentName(final long segmentId) {
        // previous format used - as a separator so if this changes in the future
        // then we should use something different.
        return name + ":" + segmentId * segmentInterval;
    }

"segmentName" is passed to RocksDB, RockDB will use it as file name to create the DB file, as we known, the ":" cannot be part of file name in Windows OS.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)