hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Bortnikov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15016) StoreServices facility in Region
Date Wed, 30 Dec 2015 21:51:49 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075447#comment-15075447

Edward Bortnikov commented on HBASE-15016:

[~eshcar], could you please attach a small doc/diagram describing the semantics of all the
counters (Active, Fluctuating, etc) how they are used by flush policy. It would be a good
intro to this patch. 

In my understanding, the idea is to have different flush threshold depending on the store
types. In the default store implementation, there are an active segment (mutable) and a snapshot
segment (immutable). In the compacting store (which is not part of this patch), there is also
a fluctuating part (pipeline), consisting of possibly multiple immutable segments. This affects
the arithmetic of when to flush, because we are interested to let the fluctuating part grow
for a while, and exploit it as buffer for redundancy elimination. The challenge is to make
HRegion flush triggering polymorphic without really understanding much about these delicacies.
That was the reason for exposing a small number of counters that the policy can deal with.

[~stack], do you have in mind a more elegant solution maybe? 

> StoreServices facility in Region
> --------------------------------
>                 Key: HBASE-15016
>                 URL: https://issues.apache.org/jira/browse/HBASE-15016
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Eshcar Hillel
>            Assignee: Eshcar Hillel
>         Attachments: HBASE-15016-V01.patch, HBASE-15016-V02.patch, HBASE-15016-V03.patch
> The default implementation of a memstore ensures that between two flushes the memstore
size increases monotonically. Supporting new memstores that store data in different formats
(specifically, compressed), or that allows to eliminate data redundancies in memory (e.g.,
via compaction), means that the size of the data stored in memory can decrease even between
two flushes. This requires memstores to have access to facilities that manipulate region counters
and synchronization.
> This subtasks introduces a new region interface -- StoreServices, through which store
components can access these facilities.

This message was sent by Atlassian JIRA

View raw message