spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Reynold Xin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-20641) Key-value store abstraction and implementation for storing application data
Date Wed, 12 Jul 2017 05:06:01 GMT

    [ https://issues.apache.org/jira/browse/SPARK-20641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083452#comment-16083452
] 

Reynold Xin edited comment on SPARK-20641 at 7/12/17 5:05 AM:
--------------------------------------------------------------

BTW why are we not using RocksDB? I saw that you just discounted by saying it has more features
than we need, but based on my experience RocksDB has a large community and has more active
development. [~dhruba] is also working on native cloud support that seems essential.

And not to mention - RocksDB can work directly with HDFS as well, whereas LevelDB can't.



was (Author: rxin):
BTW why are we not using RocksDB? I saw that you just discounted by saying it has more features
than we need, but based on my experience RocksDB has a large community and has more active
development. [~dhruba] is also working on native cloud support that seems essential.


> Key-value store abstraction and implementation for storing application data
> ---------------------------------------------------------------------------
>
>                 Key: SPARK-20641
>                 URL: https://issues.apache.org/jira/browse/SPARK-20641
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Spark Core
>    Affects Versions: 2.3.0
>            Reporter: Marcelo Vanzin
>             Fix For: 2.3.0
>
>
> See spec in parent issue (SPARK-18085) for more details.
> This task tracks adding a key-value store abstraction and initial LevelDB implementation
to be used to store application data for building the UI and REST API.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message