hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiroshi Ikeda (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14873) Problems around BoundedByteBufferPool providing direct buffers
Date Thu, 03 Dec 2015 02:58:10 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037156#comment-15037156

Hiroshi Ikeda commented on HBASE-14873:

bq. Why sir?

Sorry for my poor expression, I just complained that I prefer no waste.

As for Unsafe, this is very subtle. I just found ByteBufferUtils uses Unsafe.getInt etc. with
no restriction, but the Oracle implementation uses these methods only if the class variable
{{unaligned}} (commented as "Cached unaligned-access capability") is true, which seems to
be calculated whether the architecture is i386, x86, amd64, or x86_64. ByteBufferUtils also
uses Unsafe for on-heap buffers, but Unsafe uses native methods and it consequently prevents
VM optimization, and it might eventually wastes for on-heap buffers in long run. In addition,
Oracle seems to plan to remove Unsafe in Java 9.

bq. In other words there are situations where we should intentionally use the right buffer
in the right place.

Makes sense. I think the places where BBBP have been deployed so far make sense.

I mean, it doesn't make sense that users can switch direct/non-direct buffers to pool (in
the 1.3 branch) and developers hesitate to improve code to take advantage of direct/non-direct

bq. This issue that you filed is a dirty bug we need to fix regardless, right?

Yes, this is an absolutely serious bug that is always occurred when wrapping a cell block,
and that must be fixed in any way, regardless of my complaining :(

> Problems around BoundedByteBufferPool providing direct buffers
> --------------------------------------------------------------
>                 Key: HBASE-14873
>                 URL: https://issues.apache.org/jira/browse/HBASE-14873
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Hiroshi Ikeda
>            Assignee: Hiroshi Ikeda
>         Attachments: HBASE-14873-V2.patch, HBASE-14873.patch, HBASE-14873.patch, HBASE-14873.patch
> HBASE-13819 made BoundedByteBufferPool provide direct buffers.
> See RpcServer.java:
> {code}
> ...
> class Call implements RpcCallContext {
>   protected synchronized void setResponse(...) {
> ...
>     this.cellBlock = ipcUtil.buildCellBlock(..., reservoir);
> ...
>     bc = new BufferChain(..., this.cellBlock);
>     if (connection.useWrap) {
>       bc = wrapWithSasl(bc);
>     }
> ...
>   private BufferChain wrapWithSasl(BufferChain bc) throws IOException {
> ...
>     byte[] responseBytes = bc.getBytes();
> ...
> {code}
> {{cellBlock}} is expected to be a direct buffer retrieved from {{reservoir}} (but not
always), and {{bc}} may be composed of both direct and non-direct buffers.
> And then, see BufferChain.java:
> {code}
> byte [] getBytes() {
> ...
>     for (ByteBuffer bb: this.buffers) {
>       System.arraycopy(bb.array(), ...);
> {code}
> A direct buffer doesn't give its array, and will throw UnsupportedOperationException.
> Another problem; {{cellBlock}} is allowed to be a non-direct buffer, and after use it
will be put to {{reservoir}}, mixing direct and non-direct buffers in the pool.

This message was sent by Atlassian JIRA

View raw message