hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3551) Loaded hfile indexes occupy a good chunk of heap; look into shrinking the amount used and/or evicting unused indices
Date Tue, 22 Feb 2011 19:24:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997985#comment-12997985
] 

stack commented on HBASE-3551:
------------------------------

@Ryan Regards key size, was small in this case and not tweakable.  Same for index offset.
 Didn't want to use bigger blocks.

@Andrew So, you are suggesting that we let go of whole file not just the index.  That is probably
the better thing to do; it addresses index-size and other costs associated with keeping files
open.


> Loaded hfile indexes occupy a good chunk of heap; look into shrinking the amount used
and/or evicting unused indices
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-3551
>                 URL: https://issues.apache.org/jira/browse/HBASE-3551
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: stack
>
> I hung with a user Marc and we were looking over configs and his cluster profile up on
ec2.  One thing we noticed was that his 100+ 1G regions of two families had ~2.5G of heap
resident.  We did a bit of math and couldn't get to 2.5G so that needs looking into.  Even
still, 2.5G is a bunch of heap to give over to indices (He actually OOME'd when he had his
RS heap set to just 3G; we shouldn't OOME, we should just run slower).  It sounds like he
needs the indices loaded but still, for some cases we should drop indices for unaccessed files.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message