zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fangmin Lv (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ZOOKEEPER-3352) Use LevelDB For Backend
Date Thu, 09 May 2019 05:43:00 GMT

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-3352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836096#comment-16836096

Fangmin Lv commented on ZOOKEEPER-3352:

Had an intern done some prototype with me by building the ZooKeeper on RocksDB, the intention
is to store more data, use incremental snapshot to reduce the cost with LSM, reduce the
GC effort and data loading time, etc. It also avoids fuzzy snapshot which is pretty error-prone
and caused lots of consistency bugs in ZK. But it's likely won't give us the same throughput as
what we have now with in memory data tree and append only txn file. 

Currently, we only moved snapshot to RocksDB, the plan is to remove the in memory data tree
as well, but only cache things like children structure, metadata to retain txn prepare and
read throughput, etc.

We tried to keep the file based txn file, but disable the WAL in RocksDB, since we cannot
see obvious benefit for now.

This is still a prototype, and we didn't have time in the last year to spend more effort there,
but it will be an interesting try to see how it goes when ZooKeeper meet other backend.

[~belugabehr] what's the status of this project on your side, is it a thought or you already
have something?



> Use LevelDB For Backend
> -----------------------
>                 Key: ZOOKEEPER-3352
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3352
>             Project: ZooKeeper
>          Issue Type: New Feature
>          Components: server
>            Reporter: David Mollitor
>            Assignee: David Mollitor
>            Priority: Critical
>             Fix For: 4.0.0
> Use LevelDB for managing data stored in ZK (transaction logs and snapshots).
> https://stackoverflow.com/questions/6779669/does-leveldb-support-java

This message was sent by Atlassian JIRA

View raw message