hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liang Xie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7845) optimize hfile index key
Date Wed, 20 Mar 2013 12:53:22 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607566#comment-13607566
] 

Liang Xie commented on HBASE-7845:
----------------------------------

About HFilePerformanceEvaluation, i ran three times both with and without the change on our
internal version based on 0.94.3 modified.
I found the test result is very unstable, still not sure the system resource or benchmark
code contributed right now, so the following number just for initial refer:

without change, run1:

Running SequentialWriteBenchmark for 1000000 rows took 1588ms.
Running UniformRandomSmallScan for 1000000 rows took 4771ms.
Running UniformRandomReadBenchmark for 1000000 rows took 164284ms.
Running GaussianRandomReadBenchmark for 1000000 rows took 169258ms.
Running SequentialReadBenchmark for 1000000 rows took 678ms.

without change, run2:

Running SequentialWriteBenchmark for 1000000 rows took 1432ms.
Running UniformRandomSmallScan for 1000000 rows took 5312ms.
Running UniformRandomReadBenchmark for 1000000 rows took 152581ms.
Running GaussianRandomReadBenchmark for 1000000 rows took 168693ms.
Running SequentialReadBenchmark for 1000000 rows took 686ms.

without change, run3:

Running SequentialWriteBenchmark for 1000000 rows took 1599ms.
Running UniformRandomSmallScan for 1000000 rows took 4603ms.
Running UniformRandomReadBenchmark for 1000000 rows took 154997ms.
Running GaussianRandomReadBenchmark for 1000000 rows took 175720ms.
Running SequentialReadBenchmark for 1000000 rows took 698ms.

with change, run1:

Running SequentialWriteBenchmark for 1000000 rows took 1164ms.
Running UniformRandomSmallScan for 1000000 rows took 4639ms.
Running UniformRandomReadBenchmark for 1000000 rows took 154133ms.
Running GaussianRandomReadBenchmark for 1000000 rows took 169388ms.
Running SequentialReadBenchmark for 1000000 rows took 521ms.

with change, run2:

Running SequentialWriteBenchmark for 1000000 rows took 1806ms.
Running UniformRandomSmallScan for 1000000 rows took 4554ms.
Running UniformRandomReadBenchmark for 1000000 rows took 155959ms.
Running GaussianRandomReadBenchmark for 1000000 rows took 174015ms.
Running SequentialReadBenchmark for 1000000 rows took 702ms.

with change, run3:

Running SequentialWriteBenchmark for 1000000 rows took 1510ms.
Running UniformRandomSmallScan for 1000000 rows took 5141ms.
Running UniformRandomReadBenchmark for 1000000 rows took 147168ms.
Running GaussianRandomReadBenchmark for 1000000 rows took 166012ms.
Running SequentialReadBenchmark for 1000000 rows took 692ms.

                
> optimize hfile index key
> ------------------------
>
>                 Key: HBASE-7845
>                 URL: https://issues.apache.org/jira/browse/HBASE-7845
>             Project: HBase
>          Issue Type: Improvement
>          Components: HFile
>    Affects Versions: 0.96.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
>             Fix For: 0.98.0
>
>         Attachments: HBASE-7845.txt, HBASE-7845-v2.txt, HBASE-7845-v3.txt, HBASE-7845-v4.txt,
HBASE-7845-v5.txt, HBASE-7845-v6.txt, HBASE-7845-v7.txt, HBASE-7845-v8.txt
>
>
> Leveldb uses ByteWiseComparatorImpl::FindShortestSeparator() & FindShortSuccessor()
to reduce index key size, it would be helpful under special conditions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message