Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 784AD17A0D for ; Wed, 15 Apr 2015 04:48:26 +0000 (UTC) Received: (qmail 55037 invoked by uid 500); 15 Apr 2015 04:47:59 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 54989 invoked by uid 500); 15 Apr 2015 04:47:59 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 54977 invoked by uid 99); 15 Apr 2015 04:47:59 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Apr 2015 04:47:59 +0000 Date: Wed, 15 Apr 2015 04:47:59 +0000 (UTC) From: "Lars Hofhansl (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-13472) Polish IN_MEMORY table behavior MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-13472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495659#comment-14495659 ] Lars Hofhansl commented on HBASE-13472: --------------------------------------- I thought we had three arenas in the block cache: in memory, promoted (block has been touched again after first access), and "normal" after first load. > Polish IN_MEMORY table behavior > ------------------------------- > > Key: HBASE-13472 > URL: https://issues.apache.org/jira/browse/HBASE-13472 > Project: HBase > Issue Type: Task > Reporter: Andrew Purtell > > For a long time we've been able to support a mode of operation that keeps as much table data as possible in memory, so HBase can be used as an 'in-memory' DB with fully durable WAL and write-behind persistence of table data. However: > - There are a set of relevant schema options (IN_MEMORY, CACHE_ON_WRITE, PREFETCH_BLOCKS_ON_OPEN, block encoding), so set up isn't simple. We should have a shortcut that sets all this up in one place. I'm thinking a utility class with static helpers that configure a table descriptor with all of the needed bits. (Other ideas?) > - We don't have a safety valve. An in-memory table can become too large, where it crowds out all blockcache space for other tables on the cluster, where it falls out of blockcache and performs poorly without warning because it's become too big. Consider table quota support with an option for region size limits as % of total heap consumed by regions for a given table. Warn at soft limit. Refuse writes if over hard limit. > Follow on work can investigate options hooking up to offheap work. That's not in scope here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)