hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13817) ByteBufferOuputStream - add writeInt support
Date Tue, 02 Jun 2015 10:50:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568927#comment-14568927

Anoop Sam John commented on HBASE-13817:

We do one call to the BB to putInt. Our writes are Big endian ordered. That is why making
sure that the BB we deal with in this BBOS is BIG_ENDIAN.  This is any way the default value.
 You can see that the BoundedByteBufferPool is adjusted to always return BIG_ENDIAN BBs. Yes
it is doing so now also.. Just making it explicit.  While creating the ByteBufferOutputStream,
not passing the BB, and the BB is internally created by the BBOS, saying what is the byte
order we would like to get. This is marked Public audience and so I just added an overloaded
bq.writeInte should be writeInt method name.
My bad.. will correct
bq.Is it a good idea polluting a util with BBOS specifics as in here:
Thought this is a private audience Util and is ok.. Let me see not do this. It might code
redundant code with this check in 2,3 places

I did a JMH micro benchmark where I write int to BBOS. That gives ~20% gain. But this is micro
benchmark so in the real world case it might be much smaller.

> ByteBufferOuputStream - add writeInt support
> --------------------------------------------
>                 Key: HBASE-13817
>                 URL: https://issues.apache.org/jira/browse/HBASE-13817
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Scanners
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0
>         Attachments: HBASE-13817.patch
> While writing Cells to this stream, to make the CellBlock ByteBuffer, we do write length
of the cell as int. We use StreamUtils to do this which will write each byte one after the
other. So 4 write calls on Stream.(OutputSteam has only this support) With ByteBufferOuputStream
we have the overhead of checking for size limit and possible grow with every write call. Internally
this stream writes to a ByteBuffer. Again inside the ByteBuffer implementations there is position
limit checks.  If we do write these length as int in one go we can reduce this overhead.

This message was sent by Atlassian JIRA

View raw message