hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-15064) Possbile Bug in MultiByteBuffer
Date Tue, 05 Jan 2016 06:07:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082455#comment-15082455
] 

Anoop Sam John edited comment on HBASE-15064 at 1/5/16 6:07 AM:
----------------------------------------------------------------

In BufferedEncodedSeeker#next() we can see
{code}
public boolean next() {
      if (!currentBuffer.hasRemaining()) {
        return false;
      }
      decodeNext();
	  ...
}
{code}
So when we are the end of a block, this hasRemaining is supposed to return false and so no
more calls to decodeNext. In normal case of MBB and SBB this works fine.
When the MBB is created over the last bucket, we are returning the last dummy BB of 0 size
also.
{code}
} else {
    buffers[i] = ByteBuffer.allocate(0);
}
{code}
Because of that even when we have finished all cells from a buffer, hasRemaining() will say
true for such case and for decodeNext() we will move to the last BB which is of limit 0.
{code}
@Override
  public final boolean hasRemaining() {
    return this.curItem.hasRemaining() || this.curItemIndex < this.items.length - 1;
  }
{code}


We have to solve this by fixing ByteBufferArray#asSubBuffer by not including the last dummy
buffer of 0 size from returning back to user.



was (Author: anoop.hbase):
In BufferedEncodedSeeker#next() we can see
{code}
public boolean next() {
      if (!currentBuffer.hasRemaining()) {
        return false;
      }
      decodeNext();
	  ...
}
{code}
So when we are the end of a block, this hasRemaining is supposed to return false and so no
more calls to decodeNext. In normal case of MBB and SBB this works fine.
When the MBB is created over the last bucket, we are returning the last dummy BB of 0 size
also.
{code}
} else {
    buffers[i] = ByteBuffer.allocate(0);
}
{code}
Because of that even when we have finished all cells from a buffer, hasRemaining() will say
true for such case and for decodeNext() we will move to the last BB which is of limit 0.

We have to solve this by fixing ByteBufferArray#asSubBuffer by not including the last dummy
buffer of 0 size from returning back to user.


> Possbile Bug in MultiByteBuffer
> -------------------------------
>
>                 Key: HBASE-15064
>                 URL: https://issues.apache.org/jira/browse/HBASE-15064
>             Project: HBase
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 2.0.0
>            Reporter: deepankar
>            Assignee: Anoop Sam John
>            Priority: Critical
>
> While running the newer patches on our production system, I saw this error come couple
of times 
> {noformat}
> ipc.RpcServer: Unexpected throwable object 
> 2016-01-01 16:42:56,090 ERROR [B.defaultRpcServer.handler=20,queue=20,port=60020] ipc.RpcServer:
Unexpected throwable object 
> java.nio.BufferUnderflowException
> at java.nio.Buffer.nextGetIndex(Buffer.java:500)
> at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:249)
> at org.apache.hadoop.hbase.nio.MultiByteBuff.get(MultiByteBuff.java:494)
> at org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decode(FastDiffDeltaEncoder.java:402)

> at org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decodeNext(FastDiffDeltaEncoder.java:517)

> at org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.next(BufferedDataBlockEncoder.java:815)
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:138)
> {noformat}
> Looking at the get code 
> {code}
> if (this.curItem.remaining() == 0) {
>       if (items.length - 1 == this.curItemIndex) {
>         // means cur item is the last one and we wont be able to read a long. Throw exception
>         throw new BufferUnderflowException();
>       }
>       this.curItemIndex++;
>       this.curItem = this.items[this.curItemIndex];
>     }
> return this.curItem.get();
> {code}
> Can the new currentItem have zero elements (position == limit), does it make sense to
change the {{if}} to {{while}} ? {{while (this.curItem.remaining() == 0)}}. This logic is
repeated may make sense abstract to a new function if we plan to change to  {{if}} to {{while}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message