hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15477) Do not save 'next block header' when we cache hfileblocks
Date Tue, 22 Mar 2016 13:58:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15206396#comment-15206396

stack commented on HBASE-15477:

bq. When write in BC?


bq. Also when this block is created out of a BB read from HDFS, that BB will contain the next
block's header also at the end. 


bq. So we dont need this limiting thing at all? 

We need the limit so when the bytes-read-from-hdfs are read, we do not read into the next
header, not unless we want to explicitly get at the next blocks' header -- see the bit where
we do the stuffing of the header into the thread local at cacheNextBlockHeader

bq. Because when reading cells, we will read until block's buf.hasRemaining()

Right. But setting the limit, we will not read into the extra bytes of the next header on
the block when we do the hasRemaining.

> Do not save 'next block header' when we cache hfileblocks
> ---------------------------------------------------------
>                 Key: HBASE-15477
>                 URL: https://issues.apache.org/jira/browse/HBASE-15477
>             Project: HBase
>          Issue Type: Sub-task
>          Components: BlockCache, Performance
>            Reporter: stack
>            Assignee: stack
>         Attachments: 15366v4.patch, 15477.patch, 15477v2.patch, 15477v3.patch, 15477v3.patch,
>     When we read from HDFS, we overread to pick up the next blocks header.
>     Doing this saves a seek as we move through the hfile; we save having to
>     do an explicit seek just to read the block header every time we need to
>     read the body.  We used to read in the next header as part of the
>     current blocks buffer. This buffer was then what got persisted to
>     blockcache; so we were over-persisting wrtiting out our block plus the
>     next blocks' header (overpersisting 33 bytes). Parse of HFileBlock
>     complicated by this extra tail. Fix.

This message was sent by Atlassian JIRA

View raw message