hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elek, Marton (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-12864) Ozone: Turn off full synced write in RocksDB MetadataStore
Date Tue, 28 Nov 2017 15:20:01 GMT
Elek, Marton created HDFS-12864:
-----------------------------------

             Summary: Ozone: Turn off full synced write in RocksDB MetadataStore
                 Key: HDFS-12864
                 URL: https://issues.apache.org/jira/browse/HDFS-12864
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: ozone
            Reporter: Elek, Marton
            Assignee: Elek, Marton


Based on the suggestion from [~xyao] I created the patch to turn off fully synced write in
the rocksdb.

With this patch we measured about 10x performance gain (200 nodes cluster + 30 corona instance).

1. We don't need fully synced rocksdb writing, as HA will cover the few cases when a node
would be lost

2. Long term (IMHO) we need to add a possibility to configure any settings in rocksdb, so
it could be turned off, but the setSync(false) is a reasonable default.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message