hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "deepankar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15064) BufferUnderflowException after last Cell fetched from an HFile Block served from L2 offheap cache
Date Thu, 17 Mar 2016 00:22:33 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198473#comment-15198473
] 

deepankar commented on HBASE-15064:
-----------------------------------

I am still seeing this exception on our servers, I think I found something, what I observe
is that a couple of things, in the normal byte buffers (java.nio) the hasRemaining function
uses the current position and limit {code}     /**
     * Tells whether there are any elements between the current position and
     * the limit. </p>
     *
     * @return  <tt>true</tt> if, and only if, there is at least one element
     *          remaining in this buffer
     */
    public final boolean hasRemaining() {
        return position < limit;
    }
{code}

But in the MultiByteBuff we have the hasRemaining is not taking care of limit 
{code}
 /**
   * Returns true if there are elements between the current position and the limit
   * @return true if there are elements, false otherwise
   */
  @Override
  public final boolean hasRemaining() {
    return this.curItem.hasRemaining() || this.curItemIndex < this.items.length - 1;
  }
{code}

Also the items array is not changed in the limit(int) function, this means there could be
a scenario where the user has asked to limit at the end of first buffer, but the hasRemaining()
will still return true, Is there any flaw in my logic here ?

Also in the limit(int) function  in the MultiByteBuff function we are doing 
{code}
  // Normally the limit will try to limit within the last BB item
    int limitedIndexBegin = this.itemBeginPos[this.limitedItemIndex];
    if (limit >= limitedIndexBegin && limit < this.itemBeginPos[this.limitedItemIndex
+ 1]) {
      this.items[this.limitedItemIndex].limit(limit - limitedIndexBegin);
      return this;
    }
{code}
here I think in the if statement isn't the logic be just {noformat} if (limit  > limitedIndexBegin
 && limit < this.itemBeginPos[this.limitedItemIndex + 1]) {noformat} because if
somebody is trying to limit at the place which is exactly at the boundary of the limitIndexBuffer
then we are also including the last item which does not have any data as you are limiting
at 0 (as limit == limitedIndexBegin, which is at the boundary), But then once you have read
everything in the previous buffer if the client consults hasRemaining function this will return
again true (as curIterm < no_of_items in array) but when you actually try to read anything
we will throw BufferUnderFlowException because again the last element has no data. There is
similar issue with {{getItemIndexfunction}} when again the {{elemIndex}} matches with the
boundary

> BufferUnderflowException after last Cell fetched from an HFile Block served from L2 offheap
cache
> -------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-15064
>                 URL: https://issues.apache.org/jira/browse/HBASE-15064
>             Project: HBase
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 2.0.0
>            Reporter: deepankar
>            Assignee: Anoop Sam John
>            Priority: Critical
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15064.patch
>
>
> While running the newer patches on our production system, I saw this error come couple
of times 
> {noformat}
> ipc.RpcServer: Unexpected throwable object 
> 2016-01-01 16:42:56,090 ERROR [B.defaultRpcServer.handler=20,queue=20,port=60020] ipc.RpcServer:
Unexpected throwable object 
> java.nio.BufferUnderflowException
> at java.nio.Buffer.nextGetIndex(Buffer.java:500)
> at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:249)
> at org.apache.hadoop.hbase.nio.MultiByteBuff.get(MultiByteBuff.java:494)
> at org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decode(FastDiffDeltaEncoder.java:402)

> at org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decodeNext(FastDiffDeltaEncoder.java:517)

> at org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.next(BufferedDataBlockEncoder.java:815)
> at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:138)
> {noformat}
> Looking at the get code 
> {code}
> if (this.curItem.remaining() == 0) {
>       if (items.length - 1 == this.curItemIndex) {
>         // means cur item is the last one and we wont be able to read a long. Throw exception
>         throw new BufferUnderflowException();
>       }
>       this.curItemIndex++;
>       this.curItem = this.items[this.curItemIndex];
>     }
> return this.curItem.get();
> {code}
> Can the new currentItem have zero elements (position == limit), does it make sense to
change the {{if}} to {{while}} ? {{while (this.curItem.remaining() == 0)}}. This logic is
repeated may make sense abstract to a new function if we plan to change to  {{if}} to {{while}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message