hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tatsuya Kawano (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3455) Heap fragmentation in region server
Date Wed, 26 Jan 2011 04:23:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986840#action_12986840
] 

Tatsuya Kawano commented on HBASE-3455:
---------------------------------------

Maybe this is not an optimal solution but how about this? Forget about reclaiming individual
free spaces from old cell values, but pretend them as live spaces and let Memstore to flush.
Memstore flusher won't write old cell values to HFile as they are not referenced from the
Memstore. 

You could get a stupid smaller HFiles if you do lots of ICVs, but a minor compaction will
soon merge them into one moderate-sized HFile, so it's not going to be a problem.

> Heap fragmentation in region server
> -----------------------------------
>
>                 Key: HBASE-3455
>                 URL: https://issues.apache.org/jira/browse/HBASE-3455
>             Project: HBase
>          Issue Type: Brainstorming
>          Components: performance, regionserver
>            Reporter: Todd Lipcon
>            Priority: Critical
>         Attachments: collapse-arrays.patch, HBasefragmentation.pdf, icv-frag.png, mslab-1.txt,
parse-fls-statistics.py, with-kvallocs.png
>
>
> Stop-the-world GC pauses have long been a problem in HBase. "Concurrent mode failures"
can usually be tuned around by setting the initiating occupancy fraction low, but eventually
the heap becomes fragmented and a promotion failure occurs.
> This JIRA is to do research/experiments about the heap fragmentation issue and possible
solutions.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message