hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17338) Treat Cell data size under global memstore heap size only when that Cell can not be copied to MSLAB
Date Wed, 21 Dec 2016 19:27:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15767923#comment-15767923
] 

stack commented on HBASE-17338:
-------------------------------

Do we have to check if on or offheap MSLAB?

86	  /**
87	   * @return Whether off heap based MSLAB in place.
88	   */
89	  boolean isOffheap();

Can we not have MSLAB work same whether on or offheap?

This sort of check...

101	            // issues or even OOME.
102	            if (this.memStoreLAB != null && this.memStoreLAB.isOffheap()) {
103	              heapOverheadDelta += cellLen;
104	            }


... presumes that MSLAB is done in either of two ways. This check is done apart from the implementation.

Is there copy/paste of code (going by your dup'ing the comment?).

I need to read on why Append/Increment can't be out in offheap.

This is good stuff though [~anoop.hbase]



> Treat Cell data size under global memstore heap size only when that Cell can not be copied
to MSLAB
> ---------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-17338
>                 URL: https://issues.apache.org/jira/browse/HBASE-17338
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>    Affects Versions: 2.0.0
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0
>
>         Attachments: HBASE-17338.patch
>
>
> We have only data size and heap overhead being tracked globally.  Off heap memstore works
with off heap backed MSLAB pool.  But a cell, when added to memstore, not always getting copied
to MSLAB.  Append/Increment ops doing an upsert, dont use MSLAB.  Also based on the Cell size,
we sometimes avoid MSLAB copy.  But now we track these cell data size also under the global
memstore data size which indicated off heap size in case of off heap memstore.  For global
checks for flushes (against lower/upper watermark levels), we check this size against max
off heap memstore size.  We do check heap overhead against global heap memstore size (Defaults
to 40% of xmx)  But for such cells the data size also should be accounted under the heap overhead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message