hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14873) Problems around BoundedByteBufferPool providing direct buffers
Date Thu, 03 Dec 2015 04:08:10 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037212#comment-15037212

ramkrishna.s.vasudevan commented on HBASE-14873:

Interesting and useful information here.
bq.DirectBuffer's override method calls Unsafe's copyMemory under the limit Bits.UNSAFE_COPY_THRESHOLD,
which has been bq.introduced for scaling well, according to google. How copyFromBufferToArray
can omit the logic? Another thing I should bq.mention is that copyFromBufferToArray is forced
to use one-by-one copy for the other VM implementations, which is an almost bq.bug.
Yes true.  But our intention was to mainly avoid all those boundary checks that Oracle's impl
has. Some of the earlier tests suggested that the NIO buffer APIs were not allowing the JVM
to do proper inlining because of the boundary checks in every API it has.  the same is true
with the onheap BBs also. The aim of these Byte and BytebufferUtils was to avoid these and
directly go with Unsafe.
One thing to see is that if all the  are all onheap we just treat them as Arrays
But the problems you seem to say needs to handled in the BBUtils impl?  If so, that seems
to be an important thing. But our code handles the endian thing - then may be that is not

bq.In addition, Oracle seems to plan to remove Unsafe in Java 9.
This as we know they may remove but I think they are planning to provide some other way of
using them? There was a study internally conducted and it was found that many of the projects
(apache projects included) make use of these DBBs and Unsafe Utils quite extensively. 

> Problems around BoundedByteBufferPool providing direct buffers
> --------------------------------------------------------------
>                 Key: HBASE-14873
>                 URL: https://issues.apache.org/jira/browse/HBASE-14873
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Hiroshi Ikeda
>            Assignee: Hiroshi Ikeda
>         Attachments: HBASE-14873-V2.patch, HBASE-14873.patch, HBASE-14873.patch, HBASE-14873.patch
> HBASE-13819 made BoundedByteBufferPool provide direct buffers.
> See RpcServer.java:
> {code}
> ...
> class Call implements RpcCallContext {
>   protected synchronized void setResponse(...) {
> ...
>     this.cellBlock = ipcUtil.buildCellBlock(..., reservoir);
> ...
>     bc = new BufferChain(..., this.cellBlock);
>     if (connection.useWrap) {
>       bc = wrapWithSasl(bc);
>     }
> ...
>   private BufferChain wrapWithSasl(BufferChain bc) throws IOException {
> ...
>     byte[] responseBytes = bc.getBytes();
> ...
> {code}
> {{cellBlock}} is expected to be a direct buffer retrieved from {{reservoir}} (but not
always), and {{bc}} may be composed of both direct and non-direct buffers.
> And then, see BufferChain.java:
> {code}
> byte [] getBytes() {
> ...
>     for (ByteBuffer bb: this.buffers) {
>       System.arraycopy(bb.array(), ...);
> {code}
> A direct buffer doesn't give its array, and will throw UnsupportedOperationException.
> Another problem; {{cellBlock}} is allowed to be a non-direct buffer, and after use it
will be put to {{reservoir}}, mixing direct and non-direct buffers in the pool.

This message was sent by Atlassian JIRA

View raw message