Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id DF9E2200C40 for ; Thu, 9 Mar 2017 07:05:45 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id DE393160B86; Thu, 9 Mar 2017 06:05:45 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 34BE5160B83 for ; Thu, 9 Mar 2017 07:05:45 +0100 (CET) Received: (qmail 9875 invoked by uid 500); 9 Mar 2017 06:05:43 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 9861 invoked by uid 99); 9 Mar 2017 06:05:43 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Mar 2017 06:05:43 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 5A1EFC1D3C for ; Thu, 9 Mar 2017 06:05:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.451 X-Spam-Level: * X-Spam-Status: No, score=1.451 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_NEUTRAL=0.652] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id Q4F-55PBYY6B for ; Thu, 9 Mar 2017 06:05:42 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id F33165F64B for ; Thu, 9 Mar 2017 06:05:41 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 582A6E0593 for ; Thu, 9 Mar 2017 06:05:38 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 07B18243AA for ; Thu, 9 Mar 2017 06:05:38 +0000 (UTC) Date: Thu, 9 Mar 2017 06:05:38 +0000 (UTC) From: "stack (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-17338) Treat Cell data size under global memstore heap size only when that Cell can not be copied to MSLAB MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 09 Mar 2017 06:05:46 -0000 [ https://issues.apache.org/jira/browse/HBASE-17338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902543#comment-15902543 ] stack commented on HBASE-17338: ------------------------------- +1 on commit. Lets get up on this new basis. Nice cleanup [~anoop.hbase] (Thanks too for doc edit). > Treat Cell data size under global memstore heap size only when that Cell can not be copied to MSLAB > --------------------------------------------------------------------------------------------------- > > Key: HBASE-17338 > URL: https://issues.apache.org/jira/browse/HBASE-17338 > Project: HBase > Issue Type: Sub-task > Components: regionserver > Affects Versions: 2.0.0 > Reporter: Anoop Sam John > Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-17338.patch, HBASE-17338_V2.patch, HBASE-17338_V2.patch, HBASE-17338_V4.patch, HBASE-17338_V5.patch > > > We have only data size and heap overhead being tracked globally. Off heap memstore works with off heap backed MSLAB pool. But a cell, when added to memstore, not always getting copied to MSLAB. Append/Increment ops doing an upsert, dont use MSLAB. Also based on the Cell size, we sometimes avoid MSLAB copy. But now we track these cell data size also under the global memstore data size which indicated off heap size in case of off heap memstore. For global checks for flushes (against lower/upper watermark levels), we check this size against max off heap memstore size. We do check heap overhead against global heap memstore size (Defaults to 40% of xmx) But for such cells the data size also should be accounted under the heap overhead. -- This message was sent by Atlassian JIRA (v6.3.15#6346)