hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Buttler, David" <buttl...@llnl.gov>
Subject RE: ScannerTimeoutException when a scan enables caching, no exception when it doesn't
Date Fri, 01 Apr 2011 00:22:01 GMT
I think this is expected.  The caching means that you only get blocks of 2000 rows.  And if
you go for longer than 60 seconds between blocks, then the scanner will time out.  You could
try tuning your caching down to 100 to see if that works for a bit (although, due to variance
in the time you take for processing, you might want to give yourself a bit more cushion than
that).

The next question, of course, is whether or not this is desirable behavior.
Dave


-----Original Message-----
From: Joseph Boyd [mailto:joseph.boyd@cbsinteractive.com] 
Sent: Thursday, March 31, 2011 4:59 PM
To: user@hbase.apache.org
Subject: ScannerTimeoutException when a scan enables caching, no exception when it doesn't

We're using hbase 0.90.0 here, and I'm seeing a curious behavior with my scans.

I have some code that does a scan over a table, and for each row
returned some work to verify the data...

I set the scan up like so :
  byte[] family = Bytes.toBytes("mytable");
  Scan scan = new Scan();
  scan.setCaching(2000);
  scan.addFamily(family);

and then scan using a fairly normal looking loop:

  ResultScanner scanner = table.getScanner(scan);
  for (Result userInfoResult : scanner) {
      // do some work that takes about half a second
  }

After this code runs for 60 seconds, I get the exception below:

Exception in thread "main" java.lang.RuntimeException:
org.apache.hadoop.hbase.client.ScannerTimeoutException: 78850ms passed
since the last invocation, timeout is currently set to 60000
	at org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1213)
	at cnwk.tridentp.firehose.ShowAssetMatchStatsMain.scanForProperties(ShowAssetMatchStatsMain.java:82)
	at cnwk.tridentp.firehose.ShowAssetMatchStatsMain.generateAssetMatchStats(ShowAssetMatchStatsMain.java:56)
	at cnwk.tridentp.firehose.ShowAssetMatchStatsMain.main(ShowAssetMatchStatsMain.java:33)
Caused by: org.apache.hadoop.hbase.client.ScannerTimeoutException:
78850ms passed since the last invocation, timeout is currently set to
60000
	at org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1114)
	at org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1210)
	... 3 more
Caused by: org.apache.hadoop.hbase.UnknownScannerException:
org.apache.hadoop.hbase.UnknownScannerException: Name:
-3447615741604242626
	at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1793)
	at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1036)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
	at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:83)
	at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:38)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:999)
	at org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1100)
	... 4 more


The message seems to indicate that I haven't called scanner.next() in
a long time (> 60 seconds), which isn't true, scanner.next() is being
called more than once per second.

However, if I disable caching (comment out the scan.setCaching() line
above), or set caching to a smaller value (1000, instead of 2000), the
code runs as expected.

I'm guessing that a cache-hit during my scan isn't resetting the
timeout that leads to a ScannerTimeoutException, but that's just a
guess...

Is this expected behavior?


...joe

Mime
View raw message