hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12546) Ozone: DB listing operation performance improvement
Date Mon, 09 Oct 2017 11:19:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16196823#comment-16196823
] 

Weiwei Yang commented on HDFS-12546:
------------------------------------

During investigation, I found as long as the {{getRangeKVs}} call is used with *KeyPrefixFilter*,
it can be safely replaced with {{getSequentialRangeKVs}}. This is because entries in RocksDB
are sorted, if we filter keys with *KeyPrefixFilter* then the result must be sequential. For
example if DB has keys,

 K ~1~ , K ~2~ , K ~3~, K ~4~ , K ~5~ ...

if  K ~3~ is not with prefix "abc", then all the elements after K ~3~, e.g K ~4~ , K ~5~ will
not start with "abc".

> Ozone: DB listing operation performance improvement
> ---------------------------------------------------
>
>                 Key: HDFS-12546
>                 URL: https://issues.apache.org/jira/browse/HDFS-12546
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>
> While investigating HDFS-12506, I found there are several {{getRangeKVs}} can be replaced
by {{getSequentialRangeKVs}} to improve the performance. This JIRA is to track these improvements
with sufficient tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message