hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liang Xie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9669) off-heap BloomFilter
Date Tue, 01 Oct 2013 10:33:28 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13782794#comment-13782794
] 

Liang Xie commented on HBASE-9669:
----------------------------------

there're two paths: read and write.  for read, yes, the current Hfile contains bloom filter
related blocks and then loaded into block cache(and here still have some details need to be
clear, IIRC, for data index or meta/bloom filter data, the current off-heap impl still place
them in heap probably considering perf, just place the data block into off-heap). for write,
seems to me the "ByteBuffer bloom" from ByteBloomFilter could be a potential bomb if large
than PretenureSizeThreshold sometimes.
Anyway, it's just a raw thought right now, will put more stuff after diving into the details:)

> off-heap BloomFilter
> --------------------
>
>                 Key: HBASE-9669
>                 URL: https://issues.apache.org/jira/browse/HBASE-9669
>             Project: HBase
>          Issue Type: Wish
>          Components: util
>    Affects Versions: 0.98.0
>            Reporter: Liang Xie
>
> After lru block off-heap, the bloom filter is becoming one of the largest contributers
on java heap. it should be reasonable to have a config to off-heap the bloom filter to alleviate
gc hurt.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message