hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anastasia Braginsky (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14921) Memory optimizations
Date Sun, 14 Aug 2016 12:04:22 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15420309#comment-15420309

Anastasia Braginsky commented on HBASE-14921:

bq. Makes sense. So were you able to observe the problems that we highlighted in the previous
discussions - We got two problems one is the invalid negative size for memstore and the 'Too
many open files'?
Were you able to get this patch running with the PE tool and the configs that we had updated
in the earlier discussions?

We are yet unable to use the PE tool. Even on DefaultMemStore it doesn't work properly. We
are currently running with YCSB, stressing the writes. We didn't get the exceptions that you
mentioned, neither "negative size" nor "too many open files".
However, we were able to see the problem of "too many open files" without getting the exception.
From here, the quick fix of limiting the number of segments in the pipeline. This to be managed
by a proper policy later. 

Later today I am going to publish the patch where we have instrumentation measuring the time
it takes to do the speculative scan. Will you be able to give it a run on your PE tool? You
can by the way take a look if you are still seeing the "negative size" exception. This one
(if still exists) can later be investigated while dealing with the sizes issues.

> Memory optimizations
> --------------------
>                 Key: HBASE-14921
>                 URL: https://issues.apache.org/jira/browse/HBASE-14921
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: Eshcar Hillel
>            Assignee: Anastasia Braginsky
>         Attachments: CellBlocksSegmentInMemStore.pdf, CellBlocksSegmentinthecontextofMemStore(1).pdf,
HBASE-14921-V01.patch, HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch,
HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch, HBASE-14921-V06-CAO.patch, HBASE-14921-V08-CAO.patch,
HBASE-14921-V09-CAO.patch, InitialCellArrayMapEvaluation.pdf, IntroductiontoNewFlatandCompactMemStore.pdf,
MemStoreSizes.pdf, MemstoreItrCountissue.patch, NewCompactingMemStoreFlow.pptx
> Memory optimizations including compressed format representation and offheap allocations

This message was sent by Atlassian JIRA

View raw message