hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: ScannerTimeoutException when a scan enables caching, no exception when it doesn't
Date Fri, 01 Apr 2011 00:20:36 GMT
That's the correct guess.

J-D

On Thu, Mar 31, 2011 at 4:59 PM, Joseph Boyd
<joseph.boyd@cbsinteractive.com> wrote:
> We're using hbase 0.90.0 here, and I'm seeing a curious behavior with my scans.
>
> I have some code that does a scan over a table, and for each row
> returned some work to verify the data...
>
> I set the scan up like so :
>  byte[] family = Bytes.toBytes("mytable");
>  Scan scan = new Scan();
>  scan.setCaching(2000);
>  scan.addFamily(family);
>
> and then scan using a fairly normal looking loop:
>
>  ResultScanner scanner = table.getScanner(scan);
>  for (Result userInfoResult : scanner) {
>      // do some work that takes about half a second
>  }
>
> After this code runs for 60 seconds, I get the exception below:
>
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.hadoop.hbase.client.ScannerTimeoutException: 78850ms passed
> since the last invocation, timeout is currently set to 60000
>        at org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1213)
>        at cnwk.tridentp.firehose.ShowAssetMatchStatsMain.scanForProperties(ShowAssetMatchStatsMain.java:82)
>        at cnwk.tridentp.firehose.ShowAssetMatchStatsMain.generateAssetMatchStats(ShowAssetMatchStatsMain.java:56)
>        at cnwk.tridentp.firehose.ShowAssetMatchStatsMain.main(ShowAssetMatchStatsMain.java:33)
> Caused by: org.apache.hadoop.hbase.client.ScannerTimeoutException:
> 78850ms passed since the last invocation, timeout is currently set to
> 60000
>        at org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1114)
>        at org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1210)
>        ... 3 more
> Caused by: org.apache.hadoop.hbase.UnknownScannerException:
> org.apache.hadoop.hbase.UnknownScannerException: Name:
> -3447615741604242626
>        at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1793)
>        at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
>        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1036)
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>        at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
>        at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:83)
>        at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:38)
>        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:999)
>        at org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1100)
>        ... 4 more
>
>
> The message seems to indicate that I haven't called scanner.next() in
> a long time (> 60 seconds), which isn't true, scanner.next() is being
> called more than once per second.
>
> However, if I disable caching (comment out the scan.setCaching() line
> above), or set caching to a smaller value (1000, instead of 2000), the
> code runs as expected.
>
> I'm guessing that a cache-hit during my scan isn't resetting the
> timeout that leads to a ScannerTimeoutException, but that's just a
> guess...
>
> Is this expected behavior?
>
>
> ...joe
>

Mime
View raw message