hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiroshi Ikeda (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14873) Problems around BoundedByteBufferPool providing direct buffers
Date Thu, 03 Dec 2015 02:58:10 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037156#comment-15037156
] 

Hiroshi Ikeda commented on HBASE-14873:
---------------------------------------

bq. Why sir?

Sorry for my poor expression, I just complained that I prefer no waste.

As for Unsafe, this is very subtle. I just found ByteBufferUtils uses Unsafe.getInt etc. with
no restriction, but the Oracle implementation uses these methods only if the class variable
{{unaligned}} (commented as "Cached unaligned-access capability") is true, which seems to
be calculated whether the architecture is i386, x86, amd64, or x86_64. ByteBufferUtils also
uses Unsafe for on-heap buffers, but Unsafe uses native methods and it consequently prevents
VM optimization, and it might eventually wastes for on-heap buffers in long run. In addition,
Oracle seems to plan to remove Unsafe in Java 9.

{quote}
bq. In other words there are situations where we should intentionally use the right buffer
in the right place.

Makes sense. I think the places where BBBP have been deployed so far make sense.
{quote}

I mean, it doesn't make sense that users can switch direct/non-direct buffers to pool (in
the 1.3 branch) and developers hesitate to improve code to take advantage of direct/non-direct
buffers.

bq. This issue that you filed is a dirty bug we need to fix regardless, right?

Yes, this is an absolutely serious bug that is always occurred when wrapping a cell block,
and that must be fixed in any way, regardless of my complaining :(


> Problems around BoundedByteBufferPool providing direct buffers
> --------------------------------------------------------------
>
>                 Key: HBASE-14873
>                 URL: https://issues.apache.org/jira/browse/HBASE-14873
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Hiroshi Ikeda
>            Assignee: Hiroshi Ikeda
>         Attachments: HBASE-14873-V2.patch, HBASE-14873.patch, HBASE-14873.patch, HBASE-14873.patch
>
>
> HBASE-13819 made BoundedByteBufferPool provide direct buffers.
> See RpcServer.java:
> {code}
> ...
> class Call implements RpcCallContext {
>   protected synchronized void setResponse(...) {
> ...
>     this.cellBlock = ipcUtil.buildCellBlock(..., reservoir);
> ...
>     bc = new BufferChain(..., this.cellBlock);
>     if (connection.useWrap) {
>       bc = wrapWithSasl(bc);
>     }
> ...
>   private BufferChain wrapWithSasl(BufferChain bc) throws IOException {
> ...
>     byte[] responseBytes = bc.getBytes();
> ...
> {code}
> {{cellBlock}} is expected to be a direct buffer retrieved from {{reservoir}} (but not
always), and {{bc}} may be composed of both direct and non-direct buffers.
> And then, see BufferChain.java:
> {code}
> byte [] getBytes() {
> ...
>     for (ByteBuffer bb: this.buffers) {
>       System.arraycopy(bb.array(), ...);
> {code}
> A direct buffer doesn't give its array, and will throw UnsupportedOperationException.
> Another problem; {{cellBlock}} is allowed to be a non-direct buffer, and after use it
will be put to {{reservoir}}, mixing direct and non-direct buffers in the pool.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message