hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3106) a temporary solution of multi-thread compaction for 0.20.6
Date Sat, 16 Oct 2010 14:25:22 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12921696#action_12921696
] 

stack commented on HBASE-3106:
------------------------------

Do you have a patch Andy to post?

> a temporary solution of multi-thread compaction for 0.20.6
> ----------------------------------------------------------
>
>                 Key: HBASE-3106
>                 URL: https://issues.apache.org/jira/browse/HBASE-3106
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.20.6
>         Environment: 1master, 7 region servers, 4 * 7 clients(all clients run on region
server host), sequential put 
> configuration of each host is:
> memory: 16G (region server uses 10G)
> disk: 12T
> cpu: 8 core
>            Reporter: andychen
>
> Each region's memtable size is 64M, global memtable limit is 4G on one region server
> In heavy write scenario, especially disable WAL, sequential put  fills memtable to full
very quickly, and will generates too many storefiles in a short time.
> Then there are more and more compaction tasks put into compactionQueue, single thread
excutes compaction task too slowly, finally forms a vicious cycle.
> This will block some region's data importing for a long time
> We simplely using ThreadPoolExecutor as multi-thread compaction sulution
> (As we know, 0.90 or 0.92 will optimize memflush, compaction, open, close...etc using
multi-thread, but this magnificent version hasn't bee released yet)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message