Return-Path: <issues-return-252786-archive-asf-public=cust-asf.ponee.io@hbase.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id 08B97200B58
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Wed, 27 Jul 2016 18:33:23 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id 079BC160A6F; Wed, 27 Jul 2016 16:33:23 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id 51873160A93
	for <archive-asf-public@cust-asf.ponee.io>; Wed, 27 Jul 2016 18:33:22 +0200 (CEST)
Received: (qmail 89597 invoked by uid 500); 27 Jul 2016 16:33:21 -0000
Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:issues-help@hbase.apache.org>
List-Unsubscribe: <mailto:issues-unsubscribe@hbase.apache.org>
List-Post: <mailto:issues@hbase.apache.org>
List-Id: <issues.hbase.apache.org>
Delivered-To: mailing list issues@hbase.apache.org
Received: (qmail 89385 invoked by uid 99); 27 Jul 2016 16:33:21 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Jul 2016 16:33:21 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 0DCC32C0E09
	for <issues@hbase.apache.org>; Wed, 27 Jul 2016 16:33:21 +0000 (UTC)
Date: Wed, 27 Jul 2016 16:33:21 +0000 (UTC)
From: "ramkrishna.s.vasudevan (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12917940.1449134326000.147887.1469637201053@Atlassian.JIRA>
In-Reply-To: <JIRA.12917940.1449134326000@Atlassian.JIRA>
References: <JIRA.12917940.1449134326000@Atlassian.JIRA> <JIRA.12917940.1449134326925@arcas>
Subject: [jira] [Commented] (HBASE-14921) Memory optimizations
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
archived-at: Wed, 27 Jul 2016 16:33:23 -0000


    [ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15395945#comment-15395945 ] 

ramkrishna.s.vasudevan commented on HBASE-14921:
------------------------------------------------

bq.. What is the exact cluster configuration you are running on? Is it a single machine hosting both the master and 1 RS? What is the setting of the underlying HDFS? Again NN and 1 DN on the same machine?
Yes this is a single node system. 1NN and 1DN all running on same machine.
{code}
Xmx - 32G
<property>
  <name>hbase.regionserver.global.memstore.size</name>
  <value>0.42</value>
</property>
<property>
    <name>hbase.hregion.memstore.mslab.enabled</name>
    <value>true</value>
  </property>

<property>
    <name>hbase.hregion.memstore.chunkpool.maxsize</name>
    <value>1</value>
  </property>

<property>
    <name>hbase.hregion.memstore.chunkpool.initialsize</name>
    <value>0.5</value>
  </property>
{code}
Do you need any other information?
bq.--inmemoryCompaction flag you are using in PE is not in master? Can you open a Jira to share the patch with this supplement so we don't need to re-implement this or even better commit this code to master.
I think this is already in master.
{code}
    final String inMemoryCompaction = "--inmemoryCompaction=";
{code}
bq.How much time the experiment is expected to run (approximately)?
It should run for approx 20mins. Because we load 150G of data. 
{code}
./hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --presplit=50 --size=150 --columns=50 --valueSize=200 --writeToWAL=true --bloomFilter=NONE --inmemoryCompaction=false  randomWrite 50
{code}
Thanks for the updates.

> Memory optimizations
> --------------------
>
>                 Key: HBASE-14921
>                 URL: https://issues.apache.org/jira/browse/HBASE-14921
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: Eshcar Hillel
>            Assignee: Anastasia Braginsky
>         Attachments: CellBlocksSegmentInMemStore.pdf, CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch, HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch, HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch, HBASE-14921-V06-CAO.patch, InitialCellArrayMapEvaluation.pdf, IntroductiontoNewFlatandCompactMemStore.pdf, NewCompactingMemStoreFlow.pptx
>
>
> Memory optimizations including compressed format representation and offheap allocations



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)