hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eshcar Hillel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
Date Wed, 14 Feb 2018 17:22:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16364463#comment-16364463
] 

Eshcar Hillel commented on HBASE-18294:
---------------------------------------

Following a comment by Anoop on RB I made some additional changes. Specifically propagating
the flush type which is the result of the global pressure checks to the MemStoreFlusher deciding
which region to flush. This allows choosing the region based on its on/off heap size w.r.t.
the global heap pressure triggering the flush.

I think this also answers an earlier question by stack regarding the code in the MemStoreFlusher.

Anoop the data size tracking at the RS is redundant as you suggested, but since it is already
used in several tests I prefer to keep it.

Let's see how QA ends up.

 

> Reduce global heap pressure: flush based on heap occupancy
> ----------------------------------------------------------
>
>                 Key: HBASE-18294
>                 URL: https://issues.apache.org/jira/browse/HBASE-18294
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 3.0.0
>            Reporter: Eshcar Hillel
>            Assignee: Eshcar Hillel
>            Priority: Major
>             Fix For: 2.0.0-beta-2
>
>         Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch,
HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch,
HBASE-18294.02.patch, HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, HBASE-18294.06.patch,
HBASE-18294.07.patch, HBASE-18294.07.patch, HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch,
HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, HBASE-18294.13.patch, HBASE-18294.15.patch,
HBASE-18294.16.patch, HBASE-18294.master.01.patch, HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the store to
another threshold (that can be configured with hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size (key-value only)
to the threshold where it should compare the heap size (which includes index size, and metadata).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message