hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-11425) Cell/DBB end-to-end on the read-path
Date Wed, 14 Oct 2015 17:11:06 GMT

     [ https://issues.apache.org/jira/browse/HBASE-11425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HBASE-11425:
    Attachment: gets.png

Some coarse graphs that run YCSB workload c (total random read) running for an hour with 100
clients against a dataset that is totally cached hosted on one server. The first run is against
a RS that is using default, onheap memcache. The second is using bucketcache.

I see that the work here makes it so using the bucketcache has the same latency and throughput
(perhaps a little less throughput) as serving all from onheap (recall that in tests, buckecache
as best if there were cache misses... if you could serve all from heap, onheap had a much
nicer profile). To me, this makes it possible to run with the bucketcache all the time whether
serving all from heap or when cache misses (recall, bucketcache did better when there were
cache misses -- I have not looked to see if this work improves on what we saw previous).

More testing to follow (a redo of our block cache comparisions post might be in order).

The graphs are gc basic profile (this is CMS), gets per second, the median (the 75th and 95th
percentiles weren't showing up for some reason... need to dig in... hopefully its because
their incidence was low...), and overall loading and seeks.

Offheap puts al little more load on the system, has a better GC profile, and is slightly less

> Cell/DBB end-to-end on the read-path
> ------------------------------------
>                 Key: HBASE-11425
>                 URL: https://issues.apache.org/jira/browse/HBASE-11425
>             Project: HBase
>          Issue Type: Umbrella
>          Components: regionserver, Scanners
>    Affects Versions: 0.99.0
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>         Attachments: BenchmarkTestCode.zip, Benchmarks_Tests.docx, HBASE-11425-E2E-NotComplete.patch,
HBASE-11425.patch, Offheap reads in HBase using BBs_V2.pdf, Offheap reads in HBase using BBs_final.pdf,
gc.png, gets.png, load.png, median.png
> Umbrella jira to make sure we can have blocks cached in offheap backed cache. In the
entire read path, we can refer to this offheap buffer and avoid onheap copying.
> The high level items I can identify as of now are
> 1. Avoid the array() call on BB in read path.. (This is there in many classes. We can
handle class by class)
> 2. Support Buffer based getter APIs in cell.  In read path we will create a new Cell
with backed by BB. Will need in CellComparator, Filter (like SCVF), CPs etc.
> 3. Avoid KeyValue.ensureKeyValue() calls in read path - This make byte copy.
> 4. Remove all CP hooks (which are already deprecated) which deal with KVs.  (In read
> Will add subtasks under this.

This message was sent by Atlassian JIRA

View raw message