hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13817) ByteBufferOuputStream - add writeInt support
Date Tue, 02 Jun 2015 14:48:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569193#comment-14569193

stack commented on HBASE-13817:

bq. Our writes are Big endian ordered. 

I thought this an imposition of the system we run on rather than something we specify in our

Over in Bytes, we ask what the ordering is and use it:

      static final boolean littleEndian =

Was wondering why you don't do similar in IPCUtil.

Does bb does this internally anyways: 100	      bb.order(ByteOrder.BIG_ENDIAN);  ?

bq. Thought this is a private audience Util and is ok.. 

Yeah, that is not at issue. It was more our having the util package have to have a reference
over into the io package.

Looking at your patch, it is messy having the changes in two places rather than all in StreamUtils
but otherwise, StreamUtils starts to reference other hbase packages. Arggghhhh.  Lets go with
what you have for now. Downside is we may miss your optimization as we add Cell types. That'd
be a bummer. Maybe the StreamUtils#writeInt comes into the io package in hbase at some time?

Thanks Anoop.

20% is very nice.  Suggest you attach the jmh.

> ByteBufferOuputStream - add writeInt support
> --------------------------------------------
>                 Key: HBASE-13817
>                 URL: https://issues.apache.org/jira/browse/HBASE-13817
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Scanners
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0
>         Attachments: HBASE-13817.patch, HBASE-13817_V2.patch
> While writing Cells to this stream, to make the CellBlock ByteBuffer, we do write length
of the cell as int. We use StreamUtils to do this which will write each byte one after the
other. So 4 write calls on Stream.(OutputSteam has only this support) With ByteBufferOuputStream
we have the overhead of checking for size limit and possible grow with every write call. Internally
this stream writes to a ByteBuffer. Again inside the ByteBuffer implementations there is position
limit checks.  If we do write these length as int in one go we can reduce this overhead.

This message was sent by Atlassian JIRA

View raw message