hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1902) Let PerformanceEvaluation support setting tableName and compress algorithm
Date Mon, 12 Oct 2009 16:10:31 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-1902:
-------------------------

      Resolution: Fixed
    Hadoop Flags: [Reviewed]
          Status: Resolved  (was: Patch Available)

Thanks for the patch Schubert.  Applied to TRUNK.

I added to the PE usage these new options on commit.

On the randomizing of values, I think in the BT paper, they intentionally made it so values
were all over the place so compression had no effect on observed performance.  I get the sense
that compression was always on in their system hence they did this kinda thing in their test
code.  

I think your change where you make the values less random is fine because you must ask for
compression explicitly in PE.   It also makes it so can see how compression changes system
performance.   

> Let PerformanceEvaluation support setting tableName and compress algorithm
> --------------------------------------------------------------------------
>
>                 Key: HBASE-1902
>                 URL: https://issues.apache.org/jira/browse/HBASE-1902
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 0.20.1
>            Reporter: Schubert Zhang
>            Assignee: Schubert Zhang
>            Priority: Minor
>             Fix For: 0.21.0
>
>         Attachments: HBASE-1902-v1.patch
>
>
> 1. Current PerformanceEvaluation.java defaultly does not use compression, and always
use table name as "TestTable". Tow comand- arguments are added to support setting tableName
and compression algorithm by user. e.g.:
> (1) tableName="PE_LZO", compression="lzo"
> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=PE_LZO --compress=lzo
sequentialWrite 4
> (2) default (tabmeName="TesTable", compression="none"
> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite 4
> 2. Current "public static byte[] generateValue(final Random r)" generates bytes[] which
is too random. So the effect of compression is not obvious (e.g. there is not size reduce
for LZO).
> A new "public static byte[] generateValue(final Random r)" is provided here. We can get
4:1 compression ratio for LZO now.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message