hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13472) Polish IN_MEMORY table behavior
Date Wed, 15 Apr 2015 02:02:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495545#comment-14495545

Andrew Purtell commented on HBASE-13472:

On trunk we could go so far as to make IN_MEMORY a synonym for other attributes. 

> Polish IN_MEMORY table behavior
> -------------------------------
>                 Key: HBASE-13472
>                 URL: https://issues.apache.org/jira/browse/HBASE-13472
>             Project: HBase
>          Issue Type: Task
>            Reporter: Andrew Purtell
> For a long time we've been able to support a mode of operation that keeps as much table
data as possible in memory, so HBase can be used as an 'in-memory' DB with fully durable WAL
and write-behind persistence of table data. However:
> - There are a set of relevant schema options (IN_MEMORY, CACHE_ON_WRITE, PREFETCH_BLOCKS_ON_OPEN,
block encoding), so set up isn't simple. We should have a shortcut that sets all this up in
one place. I'm thinking a utility class with static helpers that configure a table descriptor
with all of the needed bits. (Other ideas?) 
> - We don't have a safety valve. An in-memory table can become too large, where it crowds
out all blockcache space for other tables on the cluster, where it falls out of blockcache
and performs poorly without warning because it's become too big. Consider table quota support
with an option for region size limits as % of total heap consumed by regions for a given table.
Warn at soft limit. Refuse writes if over hard limit.
> Follow on work can investigate options hooking up to offheap work. That's not in scope

This message was sent by Atlassian JIRA

View raw message