hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Qiang Tian <tian...@gmail.com>
Subject Re: scan + filter failed with OutOfOrderScannerNextException
Date Mon, 29 Sep 2014 09:55:11 GMT
How many threads in your client?

On Mon, Sep 29, 2014 at 4:05 PM, Henry Hung <YTHung1@winbond.com> wrote:

> Hi Anoop,
>
> I receive this error in client side, and pretty sure the scan failed.
> I'm using default caching, so it should be 100, right?
> About scan time out period, I will try to set it higher, probably 1 hour.
>
> BTW, I'm using hbase 0.96.0.
>
> Best regards,
> Henry
>
> -----Original Message-----
> From: Anoop John [mailto:anoop.hbase@gmail.com]
> Sent: Monday, September 29, 2014 3:13 PM
> To: user@hbase.apache.org
> Subject: Re: scan + filter failed with OutOfOrderScannerNextException
>
> Hi
>    Even when the RS throws this Exception, the client side will start a
> new Scanner and retry. U just see this in log or the scan is failing
> altogether?
> What is the caching you use on Scan?  When most of the rows are filtered
> out at server side, it takes more time to fetch and return the 'caching'
> number of rows back to client. By this time the client would have already
> timed out and sent a retry. We allow such a retry will cause not getting
> back all the results. This is a serious problem and this Exception way
> helps there.   You can try increasing the scan time out period at client
> side. This should help.
>
> -Anoop-
>
> On Mon, Sep 29, 2014 at 12:35 PM, Henry Hung <YTHung1@winbond.com> wrote:
>
> > Hi All,
> >
> > Is there a way to let scanner finish scanning all regions without
> > throwing this kind of error? I'm using scan with filter MUST_PASS_ALL,
> > and I observe that whenever the result data is smaller (let's say 10%)
> > compare to another filter with larger result (let's say 80%), it always
> failed.
> >
> > org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of
> > OutOfOrderScannerNextException: was there a rpc timeout?
> > at
> > org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:3
> > 91)
> > at
> > com.winbond.hbase.schema.mesperflogtime.MesPerfLogTimeResultSet.next(M
> > esPerfLogTimeResultSet.java:58)
> > at
> > com.winbond.hbase.schema.mesperflogtime.MesPerfLogTimeResultSet.next(M
> > esPerfLogTimeResultSet.java:1) at
> > com.winbond.hbase.executor.Program.main(Program.java:106)
> > Caused by:
> > org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException:
> > org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException:
> > Expected
> > nextCallSeq: 1 But the nextCallSeq got from client: 0;
> request=scanner_id:
> > 1594996183743944787 number_of_rows: 100 close_scanner: false
> next_call_seq:
> > 0
> > at
> > org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.
> > java:3007)
> > at
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$
> > 2.callBlockingMethod(ClientProtos.java:26929)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
> > at
> > org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
> >
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > Method) at
> > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructo
> > rAccessorImpl.java:39)
> > at
> > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCo
> > nstructorAccessorImpl.java:27) at
> > java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> > at
> > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteExcep
> > tion.java:106)
> > at
> > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteExce
> > ption.java:95)
> > at
> > org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(Proto
> > bufUtil.java:235)
> > at
> > org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.ja
> > va:197)
> > at
> > org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.ja
> > va:57)
> > at
> > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRe
> > tryingCaller.java:116)
> > at
> > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRe
> > tryingCaller.java:94)
> > at
> > org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:3
> > 43)
> > ... 3 more
> > Caused by:
> >
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException):
> > org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException:
> > Expected
> > nextCallSeq: 1 But the nextCallSeq got from client: 0;
> request=scanner_id:
> > 1594996183743944787 number_of_rows: 100 close_scanner: false
> next_call_seq:
> > 0
> > at
> > org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.
> > java:3007)
> > at
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$
> > 2.callBlockingMethod(ClientProtos.java:26929)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
> > at
> > org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
> >
> > at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1446)
> > at
> > org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.jav
> > a:1650)
> > at
> > org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation
> > .callBlockingMethod(RpcClient.java:1708)
> > at
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$
> > BlockingStub.scan(ClientProtos.java:27332)
> > at
> > org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.ja
> > va:167)
> > ... 7 more
> >
> > Best regards,
> > Henry
> >
> > ________________________________
> > The privileged confidential information contained in this email is
> > intended for use only by the addressees as indicated by the original
> > sender of this email. If you are not the addressee indicated in this
> > email or are not responsible for delivery of the email to such a
> > person, please kindly reply to the sender indicating this fact and
> > delete all copies of it from your computer and network server
> > immediately. Your cooperation is highly appreciated. It is advised
> > that any unauthorized use of confidential information of Winbond is
> > strictly prohibited; and any information in this email irrelevant to
> > the official business of Winbond shall be deemed as neither given nor
> endorsed by Winbond.
> >
>
> The privileged confidential information contained in this email is
> intended for use only by the addressees as indicated by the original sender
> of this email. If you are not the addressee indicated in this email or are
> not responsible for delivery of the email to such a person, please kindly
> reply to the sender indicating this fact and delete all copies of it from
> your computer and network server immediately. Your cooperation is highly
> appreciated. It is advised that any unauthorized use of confidential
> information of Winbond is strictly prohibited; and any information in this
> email irrelevant to the official business of Winbond shall be deemed as
> neither given nor endorsed by Winbond.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message