hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eshcar Hillel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-20542) Better heap utilization for IMC with MSLABs
Date Sun, 01 Jul 2018 18:03:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529187#comment-16529187
] 

Eshcar Hillel commented on HBASE-20542:
---------------------------------------

Thanks [~Apache9], pushed the patch to master and branch-2.

So what are the plans for testing performance of 2.1.0 and ensuing releases? Is it all in
the context of HBASE-20188? Running load (a) and then workloada and workloadc of YCSB? Would
you also consider workloadx?
The reason I ask is that we would like to suggest setting IMC as default again.

> Better heap utilization for IMC with MSLABs
> -------------------------------------------
>
>                 Key: HBASE-20542
>                 URL: https://issues.apache.org/jira/browse/HBASE-20542
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Eshcar Hillel
>            Assignee: Eshcar Hillel
>            Priority: Major
>         Attachments: HBASE-20542.branch-2.001.patch, HBASE-20542.branch-2.003.patch,
HBASE-20542.branch-2.004.patch, HBASE-20542.branch-2.005.patch, HBASE-20542.master.003.patch,
run.sh, workloada, workloadc, workloadx, workloady
>
>
> Following HBASE-20188 we realized in-memory compaction combined with MSLABs may suffer
from heap under-utilization due to internal fragmentation. This jira presents a solution to
circumvent this problem. The main idea is to have each update operation check if it will cause
overflow in the active segment *before* it is writing the new value (instead of checking the
size after the write is completed), and if it is then the active segment is atomically swapped
with a new empty segment, and is pushed (full-yet-not-overflowed) to the compaction pipeline.
Later on the IMC deamon will run its compaction operation (flatten index/merge indices/data
compaction) in the background. Some subtle concurrency issues should be handled with care.
We next elaborate on them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message