hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14921) Memory optimizations
Date Wed, 27 Jul 2016 16:33:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15395945#comment-15395945

ramkrishna.s.vasudevan commented on HBASE-14921:

bq.. What is the exact cluster configuration you are running on? Is it a single machine hosting
both the master and 1 RS? What is the setting of the underlying HDFS? Again NN and 1 DN on
the same machine?
Yes this is a single node system. 1NN and 1DN all running on same machine.
Xmx - 32G


Do you need any other information?
bq.--inmemoryCompaction flag you are using in PE is not in master? Can you open a Jira to
share the patch with this supplement so we don't need to re-implement this or even better
commit this code to master.
I think this is already in master.
    final String inMemoryCompaction = "--inmemoryCompaction=";
bq.How much time the experiment is expected to run (approximately)?
It should run for approx 20mins. Because we load 150G of data. 
./hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --presplit=50 --size=150
--columns=50 --valueSize=200 --writeToWAL=true --bloomFilter=NONE --inmemoryCompaction=false
 randomWrite 50
Thanks for the updates.

> Memory optimizations
> --------------------
>                 Key: HBASE-14921
>                 URL: https://issues.apache.org/jira/browse/HBASE-14921
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: Eshcar Hillel
>            Assignee: Anastasia Braginsky
>         Attachments: CellBlocksSegmentInMemStore.pdf, CellBlocksSegmentinthecontextofMemStore(1).pdf,
HBASE-14921-V01.patch, HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch,
HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch, HBASE-14921-V06-CAO.patch, InitialCellArrayMapEvaluation.pdf,
IntroductiontoNewFlatandCompactMemStore.pdf, NewCompactingMemStoreFlow.pptx
> Memory optimizations including compressed format representation and offheap allocations

This message was sent by Atlassian JIRA

View raw message