hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-12562) Handling memory pressure for secondary region replicas
Date Tue, 03 Mar 2015 21:31:05 GMT

     [ https://issues.apache.org/jira/browse/HBASE-12562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Enis Soztutar updated HBASE-12562:
    Attachment: hbase-12562_v1.patch

Here is a patch that implements a policy for freeing memstores by doing a "refresh store files"
if necessary. Since secondary regions cannot flush by themselves, we use this extreme measure
if needed. 

When there is global memory pressure, we have a chore to find out the best possible region
to flush. This patch extends that to find the best possible primary region and best possible
secondary region (biggest memstore), and then we compare their memstore sizes. We use a multiplier
(default 4) for the comparison because we want to flush primary regions as much as possible,
but not do refresh store files (because of semantics as detailed in the design doc). 

> Handling memory pressure for secondary region replicas
> ------------------------------------------------------
>                 Key: HBASE-12562
>                 URL: https://issues.apache.org/jira/browse/HBASE-12562
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>             Fix For: 2.0.0
>         Attachments: hbase-12562_v1.patch
> This issue will track the implementation of how to handle the memory pressure for secondary
region replicas. Since the replicas cannot flush by themselves, the region server might get
blocked or cause extensive flushing for its primary regions. The design doc attached at HBASE-11183
contains two possible solutions that we can pursue. The first one is to not allow secondary
region replicas to not flush by themselves, but instead of needed allow them to refresh their
store files on demand (which possibly allows them to drop their memstore snapshots or memstores).
The second approach is to allow the secondaries to flush to a temporary space. 
> Both have pros and cons, but for simplicity and to not cause extra write amplification,
we have implemented the first approach. More details can be found in the design doc, but we
can also discuss other options here. 

This message was sent by Atlassian JIRA

View raw message