hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17484) Add non cached version of OffheapKV for write path
Date Thu, 19 Jan 2017 05:09:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15829324#comment-15829324

ramkrishna.s.vasudevan commented on HBASE-17484:

bq.Any evidence to present to justify why we should have these new types? We've gone this
route a bunch of times in our history adding caching and then undoing it. The argument is
that reading these lengths by parsing offheap bytes is slow?
Previously this offheapKV was getting used only in read path and we found that caching it
did not have a big impact because they are all short lived objects that gets created in case
of random reads. And during a block seek caching these values helped us.
But in case of write path we add them to the memstore and we see that if we have more caching
then the flush related calculations related to blocking the incoming write requests (due to
higher limit breach) are impacted more by this caching. If you see the KeyValue case we don't
do these caching and so we try to do the same here for the write path so that we have the
same data and heap overhead for both KeyValues and OffheapKeyValues.
bq.After this patch, how many versions of KeyValue do we have?
We thought about this and this would be a question. I think after that ExtendedCell was added
we have considerably reduced the variations. Let me check on that on exact number.
bq.Do we have to have a OffheapKeyValue and Cached.
With 'Cached' I meant the an OffheapKV with more of its states cached. ByteBufferCell for
now is purely only for direct. Anything with non-direct we create KV out of it.
bq.OffheapKeyValue.FIXED_OVERHEAD + Bytes.SIZEOF_INT + Bytes.SIZEOF_SHORT; .... is that a
double ''?
I think it is ok as we cache the keyLen (an int) and rowLen (a short) here. 

> Add non cached version of OffheapKV for write path
> --------------------------------------------------
>                 Key: HBASE-17484
>                 URL: https://issues.apache.org/jira/browse/HBASE-17484
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>         Attachments: HBASE-17484.patch
> After running lot of different performance tests for various scenarios and with multi
threads we thought that is  better to have a version of OffheapKV in write path that does
not cache anything and its fixed_overhead is equal to that in KeyValue. 

This message was sent by Atlassian JIRA

View raw message