hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13472) Polish IN_MEMORY table behavior
Date Mon, 27 Apr 2015 04:48:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513525#comment-14513525
] 

Anoop Sam John commented on HBASE-13472:
----------------------------------------

bq. Consider table quota support with an option for region size limits as % of total heap
consumed by regions for a given table. Warn at soft limit. Refuse writes if over hard limit.
Ya.. I was thinking about this as a work after the offheaping Jira. When we can have HBase
RS with more and more cache, we would need better tuning of the size shares for different
tables.  So Quotas can help here.

We have L1 and L2 cache now (In 2 modes).  I was thinking whether we can extend this.  We
have L1 cache in on heap and L2 in Offheap (BC) and a L3 in BC file mode (SSDs). Out of scope
of this Jira.  Just noting down here as here we speak about keeping more data in cache. We
would need quota control and table level priority control....  cc [~stack]

> Polish IN_MEMORY table behavior
> -------------------------------
>
>                 Key: HBASE-13472
>                 URL: https://issues.apache.org/jira/browse/HBASE-13472
>             Project: HBase
>          Issue Type: Task
>            Reporter: Andrew Purtell
>
> For a long time we've been able to support a mode of operation that keeps as much table
data as possible in memory, so HBase can be used as an 'in-memory' DB with fully durable WAL
and write-behind persistence of table data. However:
> - There are a set of relevant schema options (IN_MEMORY, CACHE_ON_WRITE, PREFETCH_BLOCKS_ON_OPEN,
block encoding), so set up isn't simple. We should have a shortcut that sets all this up in
one place. I'm thinking a utility class with static helpers that configure a table descriptor
with all of the needed bits. (Other ideas?) 
> - We don't have a safety valve. An in-memory table can become too large, where it falls
out of blockcache and performs poorly without warning because it's become too big. Consider
table quota support with an option for region size limits as % of total heap consumed by regions
for a given table. Warn at soft limit. Refuse writes if over hard limit.
> Follow on work can investigate options hooking up to offheap work. That's not in scope
here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message