Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of jdcryans@gmail.com designates
 209.85.214.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=gZ1cPZ3hAjgBy0VGL28IUST3MWx/etVysNU3FVBoMzYYt0nz6I1YuMdwRqu0U6Fw67
         GrwF1RROmioty9cpvS7BmdqqZuHk2VAwj6zBeO/gCj4/epA2np+TPTDV8RA+fTILKdCm
         ZR7jIZWK/JYluoie1fur4NtrISZOwNdgRMDJA=
MIME-Version: 1.0
Sender: jdcryans@gmail.com
In-Reply-To: <AANLkTinZagm1qWwLynOkmiKRWAA-3Q6hrPVyTQb_+oL0@mail.gmail.com>
References: <AANLkTinZagm1qWwLynOkmiKRWAA-3Q6hrPVyTQb_+oL0@mail.gmail.com>
Date: Thu, 31 Mar 2011 17:20:36 -0700
Message-ID: <AANLkTimbVmCTRP2+9WMWdoZf3JsP=H-M8kQiPkkT+zwe@mail.gmail.com>
Subject: Re: ScannerTimeoutException when a scan enables caching, no exception
 when it doesn't
From: Jean-Daniel Cryans <jdcryans@apache.org>
To: user@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

That's the correct guess.

J-D

On Thu, Mar 31, 2011 at 4:59 PM, Joseph Boyd
<joseph.boyd@cbsinteractive.com> wrote:
> We're using hbase 0.90.0 here, and I'm seeing a curious behavior with my =
scans.
>
> I have some code that does a scan over a table, and for each row
> returned some work to verify the data...
>
> I set the scan up like so :
> =A0byte[] family =3D Bytes.toBytes("mytable");
> =A0Scan scan =3D new Scan();
> =A0scan.setCaching(2000);
> =A0scan.addFamily(family);
>
> and then scan using a fairly normal looking loop:
>
> =A0ResultScanner scanner =3D table.getScanner(scan);
> =A0for (Result userInfoResult : scanner) {
> =A0 =A0 =A0// do some work that takes about half a second
> =A0}
>
> After this code runs for 60 seconds, I get the exception below:
>
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.hadoop.hbase.client.ScannerTimeoutException: 78850ms passed
> since the last invocation, timeout is currently set to 60000
> =A0 =A0 =A0 =A0at org.apache.hadoop.hbase.client.HTable$ClientScanner$1.h=
asNext(HTable.java:1213)
> =A0 =A0 =A0 =A0at cnwk.tridentp.firehose.ShowAssetMatchStatsMain.scanForP=
roperties(ShowAssetMatchStatsMain.java:82)
> =A0 =A0 =A0 =A0at cnwk.tridentp.firehose.ShowAssetMatchStatsMain.generate=
AssetMatchStats(ShowAssetMatchStatsMain.java:56)
> =A0 =A0 =A0 =A0at cnwk.tridentp.firehose.ShowAssetMatchStatsMain.main(Sho=
wAssetMatchStatsMain.java:33)
> Caused by: org.apache.hadoop.hbase.client.ScannerTimeoutException:
> 78850ms passed since the last invocation, timeout is currently set to
> 60000
> =A0 =A0 =A0 =A0at org.apache.hadoop.hbase.client.HTable$ClientScanner.nex=
t(HTable.java:1114)
> =A0 =A0 =A0 =A0at org.apache.hadoop.hbase.client.HTable$ClientScanner$1.h=
asNext(HTable.java:1210)
> =A0 =A0 =A0 =A0... 3 more
> Caused by: org.apache.hadoop.hbase.UnknownScannerException:
> org.apache.hadoop.hbase.UnknownScannerException: Name:
> -3447615741604242626
> =A0 =A0 =A0 =A0at org.apache.hadoop.hbase.regionserver.HRegionServer.next=
(HRegionServer.java:1793)
> =A0 =A0 =A0 =A0at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown So=
urce)
> =A0 =A0 =A0 =A0at sun.reflect.DelegatingMethodAccessorImpl.invoke(Delegat=
ingMethodAccessorImpl.java:25)
> =A0 =A0 =A0 =A0at java.lang.reflect.Method.invoke(Method.java:597)
> =A0 =A0 =A0 =A0at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseR=
PC.java:570)
> =A0 =A0 =A0 =A0at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBa=
seServer.java:1036)
> =A0 =A0 =A0 =A0at sun.reflect.NativeConstructorAccessorImpl.newInstance0(=
Native Method)
> =A0 =A0 =A0 =A0at sun.reflect.NativeConstructorAccessorImpl.newInstance(N=
ativeConstructorAccessorImpl.java:39)
> =A0 =A0 =A0 =A0at sun.reflect.DelegatingConstructorAccessorImpl.newInstan=
ce(DelegatingConstructorAccessorImpl.java:27)
> =A0 =A0 =A0 =A0at java.lang.reflect.Constructor.newInstance(Constructor.j=
ava:513)
> =A0 =A0 =A0 =A0at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRe=
moteException(RemoteExceptionHandler.java:96)
> =A0 =A0 =A0 =A0at org.apache.hadoop.hbase.client.ScannerCallable.call(Sca=
nnerCallable.java:83)
> =A0 =A0 =A0 =A0at org.apache.hadoop.hbase.client.ScannerCallable.call(Sca=
nnerCallable.java:38)
> =A0 =A0 =A0 =A0at org.apache.hadoop.hbase.client.HConnectionManager$HConn=
ectionImplementation.getRegionServerWithRetries(HConnectionManager.java:999=
)
> =A0 =A0 =A0 =A0at org.apache.hadoop.hbase.client.HTable$ClientScanner.nex=
t(HTable.java:1100)
> =A0 =A0 =A0 =A0... 4 more
>
>
> The message seems to indicate that I haven't called scanner.next() in
> a long time (> 60 seconds), which isn't true, scanner.next() is being
> called more than once per second.
>
> However, if I disable caching (comment out the scan.setCaching() line
> above), or set caching to a smaller value (1000, instead of 2000), the
> code runs as expected.
>
> I'm guessing that a cache-hit during my scan isn't resetting the
> timeout that leads to a ScannerTimeoutException, but that's just a
> guess...
>
> Is this expected behavior?
>
>
> ...joe
>