hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-12332) [mob] use filelink instead of retry when resolving mobfiles
Date Thu, 22 Jan 2015 02:36:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14286834#comment-14286834
] 

Jonathan Hsieh edited comment on HBASE-12332 at 1/22/15 2:36 AM:
-----------------------------------------------------------------

Hi [~jiajia], I'm back and will be actively working on this.  

I did a performance test wit the hfilelink version compared to prevoius the version, and confirmed
that the perf difference is fairly large.

Using load test tool, with mob enabled,  here's my results of executing the following steps.

* start hbase
* bin/hbase ltt -init_only
* in hbase shell alter list[0], {NAME => ‘test_cf’, IS_MOB=>true, MOB_THRESHOLD=8
}
* // preload - time bin/hbase ltt -write 1:8192:10 -num_keys 20000 
* // load - time bin/hbase ltt -write 1:8192:10 -num_keys 200000
* // read - time bin/hbase ltt -skip_init -read 5:10 -num_keys 200000
* truncate table hbase 

with HBASE-12332 v4 - the load and read phases took 2m28s and 2m05s respectively.
without HBASE-12332 v4 - we had 1m25s and 41.5s respectively. 

I agree that v4 is too slow. 

I'll take a look at your v5.




was (Author: jmhsieh):
Hi [~jiajia], I'm back and will be actively working on this.  

I did a performance test wit the hfilelink version compared to prevoius the version, and confirmed
that the perf difference is fairly large.

Using load test tool, with mob enabled,  here's my results of executing the following steps.

* start hbase
* bin/hbase ltt -init_only
* in hbase shell alter list[0], {NAME => ‘test_cf’, IS_MOB=>true, MOB_THRESHOLD=8
}
* // preload - time bin/hbase ltt -write 1:8192:10 -num_keys 20000 
* // load - time bin/hbase ltt -write 1:8192:10 -num_keys 200000
* // read - time bin/hbase ltt -skip_init -read 5:10 -num_keys 200000
* truncate table hbase 

with HBASE-12332 v4 - the load and read phases took 2m28s and 2m05s respectively.
without HBASE-12332 v4 - we had 1m25s and 41.5s respectively. 

The I agree that v4 is too slow. 

I'll take a look at your v5.



> [mob] use filelink instead of retry when resolving mobfiles
> -----------------------------------------------------------
>
>                 Key: HBASE-12332
>                 URL: https://issues.apache.org/jira/browse/HBASE-12332
>             Project: HBase
>          Issue Type: Sub-task
>          Components: mob
>    Affects Versions: hbase-11339
>            Reporter: Jonathan Hsieh
>             Fix For: hbase-11339
>
>         Attachments: HBASE-12332-V1.diff, HBASE-12332-V2.patch, HBASE-12332-V3.patch,
HBASE-12332-V5.patch, hbase-12332.link.v4.patch, hbase-12332.patch
>
>
> in the snapshot code, hmobstore was modified to traverse an hfile link to a mob.   Ideally
this should use the transparent filelink code to read the data.
> Also there will likely be some issues with the mob file cache with these links.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message