hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3455) Heap fragmentation in region server
Date Wed, 26 Jan 2011 21:34:47 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12987235#action_12987235
] 

Jonathan Gray commented on HBASE-3455:
--------------------------------------

A slight aside, but in a heavy concurrency increment-only use case we are CPU bound with 80%
or so being used in MemStore / CSLM operations.  We are playing with different ways of optimizing
the MemStore+Increments use case.  Today, it takes between 3 and 4 CSLM operations for each
increment.  We should be able to get this down to 1 or 2, but in all likelihood this would
be an increment-only optimization.  Rather than modifying the original bytes in place, we
are making a new byte[] and modifying the KV to point at the new byte[].  This keeps the modification
atomic and prevents build-up (and costly removal) of KVs.

Will file a JIRA for this.

> Heap fragmentation in region server
> -----------------------------------
>
>                 Key: HBASE-3455
>                 URL: https://issues.apache.org/jira/browse/HBASE-3455
>             Project: HBase
>          Issue Type: Brainstorming
>          Components: performance, regionserver
>            Reporter: Todd Lipcon
>            Priority: Critical
>         Attachments: collapse-arrays.patch, HBasefragmentation.pdf, icv-frag.png, mslab-1.txt,
parse-fls-statistics.py, with-kvallocs.png
>
>
> Stop-the-world GC pauses have long been a problem in HBase. "Concurrent mode failures"
can usually be tuned around by setting the initiating occupancy fraction low, but eventually
the heap becomes fragmented and a promotion failure occurs.
> This JIRA is to do research/experiments about the heap fragmentation issue and possible
solutions.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message