hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2765) Add leveldb-based implementation for RMStateStore
Date Fri, 21 Nov 2014 19:41:33 GMT

    [ https://issues.apache.org/jira/browse/YARN-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221326#comment-14221326

Zhijie Shen commented on YARN-2765:

bq. I knew rocksdb could be used as a cache of data that came from HDFS or could be backed-up
to HDFS, but I didn't think it could read/write directly to it as part of normal operations.

Hm... I should have wrongly understand the feature. Thanks for correction.

One question about the patch: Why is it necessary to try create the DB with {{options.createIfMissing(false);}}
and then {{options.createIfMissing(true);}} if it fails at first attempt? Can't we do one
"create if missing"?

> Add leveldb-based implementation for RMStateStore
> -------------------------------------------------
>                 Key: YARN-2765
>                 URL: https://issues.apache.org/jira/browse/YARN-2765
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>         Attachments: YARN-2765.patch, YARN-2765v2.patch
> It would be nice to have a leveldb option to the resourcemanager recovery store. Leveldb
would provide some benefits over the existing filesystem store such as better support for
atomic operations, fewer I/O ops per state update, and far fewer total files on the filesystem.

This message was sent by Atlassian JIRA

View raw message