Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D611B200B52 for ; Mon, 25 Jul 2016 22:02:22 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id D49F9160A67; Mon, 25 Jul 2016 20:02:22 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 2850A160A7D for ; Mon, 25 Jul 2016 22:02:22 +0200 (CEST) Received: (qmail 64667 invoked by uid 500); 25 Jul 2016 20:02:20 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 64592 invoked by uid 99); 25 Jul 2016 20:02:20 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 25 Jul 2016 20:02:20 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id B02072C029E for ; Mon, 25 Jul 2016 20:02:20 +0000 (UTC) Date: Mon, 25 Jul 2016 20:02:20 +0000 (UTC) From: "Anastasia Braginsky (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-14921) Memory optimizations MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 25 Jul 2016 20:02:23 -0000 [ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392571#comment-15392571 ] Anastasia Braginsky commented on HBASE-14921: --------------------------------------------- First, I agree that the sizing issue is ugly and need to be improved. Which is partially done in this patch and I planned to improve it further. However, I think it is unnecessary and not urgent, to open another JIRA for this fix. This is not an issue of rebase only, because we take the code in two different directions. We could live with the code as is (or at least could see the final outcome of 14921) and later we could agree how to arrange the sizes (if what we have is not good enough)... Your two concerns are very clear. 1. The flattening without compaction is causing many small segments in pipeline, and they are not flushed all together. 2. The issue of compaction prediction cost. Please correct me if I am wrong. We understand those concerns. There is no argument that your first concern will be fixed. For your second concern we are going to benchmark it ourselves with PE. > Memory optimizations > -------------------- > > Key: HBASE-14921 > URL: https://issues.apache.org/jira/browse/HBASE-14921 > Project: HBase > Issue Type: Sub-task > Affects Versions: 2.0.0 > Reporter: Eshcar Hillel > Assignee: Anastasia Braginsky > Attachments: CellBlocksSegmentInMemStore.pdf, CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch, HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch, HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch, HBASE-14921-V06-CAO.patch, InitialCellArrayMapEvaluation.pdf, IntroductiontoNewFlatandCompactMemStore.pdf > > > Memory optimizations including compressed format representation and offheap allocations -- This message was sent by Atlassian JIRA (v6.3.4#6332)