hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions
Date Fri, 06 Oct 2017 06:20:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16194201#comment-16194201
] 

Anoop Sam John commented on HBASE-16417:
----------------------------------------

bq.So since these are in-memory compactions the effect of HDD/SSDs does it come from the fact
as how fast these segments in the pipeline are released after flushes?
Looks like that is based on the fact that how much we delay flush of certain data and so possibly
reading back most of the data from memory only and not from HDD/SSD files.   The more eager
way we do in memory flush, the lesser will be the heap size of the memstore and so the flush.
 So these tests were done with changing back to the old way of per region flush decision based
on heap size NOT on data size?
2% seems very much eager.  The more gain u r seeing because the data size is so small.  The
Key + Data size comes much lesser than the overhead because of CSLM addition.  So in effect
each flatten reducing the heap size occupancy almost by half.  The more the data size, the
lesser will be the gain.   To have a fair eval what should be the val size to used?  Any suggestions
[~stack]?    
Nice tests and detailed report.

> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>
>                 Key: HBASE-16417
>                 URL: https://issues.apache.org/jira/browse/HBASE-16417
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Anastasia Braginsky
>            Assignee: Eshcar Hillel
>             Fix For: 3.0.0
>
>         Attachments: HBASE-16417.01.patch, HBASE-16417 - Adaptive Compaction Policy -
20171001.pdf, HBASE-16417-benchmarkresults-20161101.pdf, HBASE-16417-benchmarkresults-20161110.pdf,
HBASE-16417-benchmarkresults-20161123.pdf, HBASE-16417-benchmarkresults-20161205.pdf, HBASE-16417-benchmarkresults-20170309.pdf,
HBASE-16417-benchmarkresults-20170317.pdf, HBASE-16417 - parameter tuning - 20171001.pdf,
HBASE-16417-V01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message