hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15555) Memory Management
Date Tue, 29 Mar 2016 05:53:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215490#comment-15215490

Anoop Sam John commented on HBASE-15555:

bq.+ With offheap allocations, less GC but it runs for longer?
It is not like when offheap we do have longer GC pause.   It is when we pool the buffer. 
  We tried it.
Right now, when write request comes, we allocate new buffer and read data into it.  We read
cells from it and add to memstore. We have MSLAB copy just before doing the memstore write.
We change the RPC layer to avoid allocate of new buffer every time.  Instead , we use BBs
from the pool (BoundedByteBufferPool) and read  req into it.  After this change the #GCs (young
GC) become half but the avg pause is increased..  So there is not much diff with net GC pause
time  (Observed for some 5 mns)..   Every GC pause time bit more than the trunk case

> Memory Management
> -----------------
>                 Key: HBASE-15555
>                 URL: https://issues.apache.org/jira/browse/HBASE-15555
>             Project: HBase
>          Issue Type: Umbrella
>            Reporter: stack
>         Attachments: download.svg
> Umbrella issue on memory management. One-stop shop to learn how we're doing it, tenets,
and work to be done
> Subtasks include:
> + Map of memory allocation and recycling landscape
> + Where are we allocating when we could be recycling
> + MSLAB and G1GC don't go together?
> + Enable MSLAB pool always?
> + With offheap allocations, less GC but it runs for longer?
> + Better heuristics recycling: e.g. HBASE-15525 OutOfMemory could occur when using BoundedByteBufferPool
during RPC bursts
> + See if can do DFSClient buffer reuse: HBASE-15506 FSDataOutputStream.write() allocates
new byte buffer on each operation
> What else?

This message was sent by Atlassian JIRA

View raw message