hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15554) StoreFile$Writer.appendGeneralBloomFilter generates extra KV
Date Mon, 08 Aug 2016 08:31:20 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

ramkrishna.s.vasudevan updated HBASE-15554:
    Attachment: HBASE-15554_9.patch

Updated patch. All the duplicates have been removed in the Hash.java class. A generic HashByteExtractor
is used whose subclasses can work with byte[], BB and cell. Currently the ByteArrayHashByteExtractor
and CellBasedExtractor are used. Once this is committed can do the changes for the Read path
such a way that the cell can be used directly so that there is no byte[] version.
The suggested Iterable way I found it is not suitable here because the Hash algo knows which
byte to be extracted from which offset. The offset calculation the algo does. We only need
to ensure we fetch that byte from that offset from the underlying structure. 
CellUtil has getRowOffset, getQualOffset and getFamOffset but all are marked Private. Could
not find a suitable place to move them as there is no Util for Bloom in hbase-common.

> StoreFile$Writer.appendGeneralBloomFilter generates extra KV
> ------------------------------------------------------------
>                 Key: HBASE-15554
>                 URL: https://issues.apache.org/jira/browse/HBASE-15554
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Performance
>            Reporter: Vladimir Rodionov
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>         Attachments: HBASE-15554.patch, HBASE-15554_3.patch, HBASE-15554_4.patch, HBASE-15554_6.patch,
HBASE-15554_7.patch, HBASE-15554_9.patch
> Accounts for 10% memory allocation in compaction thread when BloomFilterType is ROWCOL.

This message was sent by Atlassian JIRA

View raw message