hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11425) Cell/DBB end-to-end on the read-path
Date Tue, 14 Apr 2015 07:20:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493712#comment-14493712

stack commented on HBASE-11425:

Took another read of the doc (and above comments on it).

#  (Continuing from comments above), suggest adding to the doc estimation of how many extra
objects will be made going this route and  Vladimir's grid to show what you are focused on.

# Did you fellas have a look at how others do offheaping or if there were libs you could have
made use of? Would have been good to include notes on your findings in here.
# The section on hasArray (if hasArray is false, it seems to imply hasByteBuffer is true)
and the discussion of added APIs and when they come into effect and when they throw unsupported
exceptions will need a rewrite in light of feedback above and review of recent patches (API
method names I think we've cleaned up too).
# Sounds like you fellas looked at netty ByteBuf too. Add in your findings I'd say.
# Would have liked to have more detail around the RPC findings. You think it could be different
now we have buffer reuse? We could save making a cellblock?
# Looking at diagrams for perf, I can't tell if more is better or not. Suggest you write up
a summary of what the diagrams are showing.
# This feature when on, will be for whole server, right? Can't do by table or region, right?

Thanks lads.

> Cell/DBB end-to-end on the read-path
> ------------------------------------
>                 Key: HBASE-11425
>                 URL: https://issues.apache.org/jira/browse/HBASE-11425
>             Project: HBase
>          Issue Type: Umbrella
>          Components: regionserver, Scanners
>    Affects Versions: 0.99.0
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>         Attachments: HBASE-11425-E2E-NotComplete.patch, HBASE-11425.patch, Offheap reads
in HBase using BBs_V2.pdf, Offheap reads in HBase using BBs_final.pdf
> Umbrella jira to make sure we can have blocks cached in offheap backed cache. In the
entire read path, we can refer to this offheap buffer and avoid onheap copying.
> The high level items I can identify as of now are
> 1. Avoid the array() call on BB in read path.. (This is there in many classes. We can
handle class by class)
> 2. Support Buffer based getter APIs in cell.  In read path we will create a new Cell
with backed by BB. Will need in CellComparator, Filter (like SCVF), CPs etc.
> 3. Avoid KeyValue.ensureKeyValue() calls in read path - This make byte copy.
> 4. Remove all CP hooks (which are already deprecated) which deal with KVs.  (In read
> Will add subtasks under this.

This message was sent by Atlassian JIRA

View raw message