hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15077) Support OffheapKV write in compaction with out copying data on heap
Date Fri, 08 Jan 2016 02:10:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088585#comment-15088585

Anoop Sam John commented on HBASE-15077:

Only int write is what we need now. As if we need later, we can add then. These are our private
ByteBufferSupportingOutputStream   seems better name. ok
Performance numbers directly taking is bit difficult. Because this will be an issue for compaction
only now. That too when that compacted file block comes from offheap L2 cache.  Later on when
we are with write path offheap, it will be more impacting and always in flushes too.
Yes we have 2 stream here. It was like a java io ByteArrayOS wrapped by a DataOS.  We need
to pass a DataOS only to other areas like DBE etc.
So here I have created both versions with supporting BB apis.
Within new DataOS, we have again another stream to which we have to write BB bytes and so
we have to call the BBUtil.

Sorry I forgot to copy the details from that jira. Let me do now.
No only that line which calc the Max array size is copied.  We do have same line with same
comment in another class also.I just copied from that :-)
bq.Elsewhere we say ByteBufferedCell, etc. Why then do ByteBufferSupportedDataOutputStream
and not ByteBufferedDOS?
When we say ByteBuffered it is implicit that it is backed by a BB.  Here it is different.
 ByteBufferSupportedDataOutputStream may not be backed  by BB it might be backed by array
only.  But it supports the BB write APIs. Am I making a point?

You want to me write a test to make sure it works as we expect? Not sure how. Any suggestion?

> Support OffheapKV write in compaction with out copying data on heap
> -------------------------------------------------------------------
>                 Key: HBASE-15077
>                 URL: https://issues.apache.org/jira/browse/HBASE-15077
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver, Scanners
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0
>         Attachments: HBASE-15077.patch
> HBASE-14832  is not enough to handle this.  Doing the remaining needed here.

This message was sent by Atlassian JIRA

View raw message