hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14921) Memory optimizations
Date Wed, 27 Jul 2016 16:33:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15395945#comment-15395945
] 

ramkrishna.s.vasudevan commented on HBASE-14921:
------------------------------------------------

bq.. What is the exact cluster configuration you are running on? Is it a single machine hosting
both the master and 1 RS? What is the setting of the underlying HDFS? Again NN and 1 DN on
the same machine?
Yes this is a single node system. 1NN and 1DN all running on same machine.
{code}
Xmx - 32G
<property>
  <name>hbase.regionserver.global.memstore.size</name>
  <value>0.42</value>
</property>
<property>
    <name>hbase.hregion.memstore.mslab.enabled</name>
    <value>true</value>
  </property>

<property>
    <name>hbase.hregion.memstore.chunkpool.maxsize</name>
    <value>1</value>
  </property>

<property>
    <name>hbase.hregion.memstore.chunkpool.initialsize</name>
    <value>0.5</value>
  </property>
{code}
Do you need any other information?
bq.--inmemoryCompaction flag you are using in PE is not in master? Can you open a Jira to
share the patch with this supplement so we don't need to re-implement this or even better
commit this code to master.
I think this is already in master.
{code}
    final String inMemoryCompaction = "--inmemoryCompaction=";
{code}
bq.How much time the experiment is expected to run (approximately)?
It should run for approx 20mins. Because we load 150G of data. 
{code}
./hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --presplit=50 --size=150
--columns=50 --valueSize=200 --writeToWAL=true --bloomFilter=NONE --inmemoryCompaction=false
 randomWrite 50
{code}
Thanks for the updates.

> Memory optimizations
> --------------------
>
>                 Key: HBASE-14921
>                 URL: https://issues.apache.org/jira/browse/HBASE-14921
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: Eshcar Hillel
>            Assignee: Anastasia Braginsky
>         Attachments: CellBlocksSegmentInMemStore.pdf, CellBlocksSegmentinthecontextofMemStore(1).pdf,
HBASE-14921-V01.patch, HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch,
HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch, HBASE-14921-V06-CAO.patch, InitialCellArrayMapEvaluation.pdf,
IntroductiontoNewFlatandCompactMemStore.pdf, NewCompactingMemStoreFlow.pptx
>
>
> Memory optimizations including compressed format representation and offheap allocations



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message