hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cedric Ho" <cedric...@gmail.com>
Subject Re: HBase UnknownScannerException
Date Tue, 30 Oct 2007 01:39:38 GMT
Hi Jim,

Thanks for the explanation, it's very helpful indeed =)

So if i understand correctly, whenever we open a scanner to perform
some batch operations on the table, there should not be any other
updates to the same table.

I suppose than it is also possible to manually catch the
UnknownScannerException in the middle of a long batch operation and
resume the operation by opening a new scanner starting from the row
key that just failed.

Do you think it is possible to handle this transparently in the
scanner.next() operation ?


Cheers,
Cedric



On 10/30/07, Jim Kellerman <jim@powerset.com> wrote:
> UnknownScannerException is thrown by the region server if a scanner request (next or
close) is called with either a bogus scanner id (unlikely since the scanner id is hidden from
the client application) or if the scanner's lease has expired.
>
> Every client request is given a lease by the region server. If the lease times out, the
region server thinks that the client has gone away and it can clean up any resources being
held for that request. Lease timeouts almost always occur on operations related to scanners.
If a scanner's lease times out, UnknownScannerException will be thrown if the client issues
a scanner request after the timeout because the region server has already cleaned up the resources
associated with the scanner. (The default lease timeout for client requests is 30 seconds,
so it is very likely that this is the situation you are running into).
>
> If you attempt to do an update to the same table that you have a scanner open on, the
update will stall waiting for the scanner to complete. So a loop which does
>
> while (scanner.next(key, val)) {
>   if (key.equals(something) || val.equals(somethingElse)) {
>     table.startUpdate(...)
>     table.put(...)
>     table.commit(...)
>   }
> }
>
> will not work because the update will wait for the scanner to finish, (which it won't),
and both the update and the next scanner.next should both fail.
>
> Hope that helps.
> ---
> Jim Kellerman, Senior Engineer; Powerset
> jim@powerset.com
>
>
> > -----Original Message-----
> > From: Cedric Ho [mailto:cedric.ho@gmail.com]
> > Sent: Thursday, October 25, 2007 7:58 PM
> > To: hadoop-user@lucene.apache.org
> > Subject: HBase UnknownScannerException
> >
> > Hi,
> >
> > I was trying HBase from the 0.15 branch. And was doing:
> >
> > HScannerInterface s = table.obtainScanner(...)
> > while(s.next(key, val)) {
> >     ....
> > }
> >
> > And encounter the following exception whenever I spent too
> > much time in the while loop, (e.g. > 30 ~ 120 seconds). I am
> > also doing insert in the same table in another process. But
> > the records they are working on are definitely non-overlapping.
> >
> > org.apache.hadoop.ipc.RemoteException:
> > org.apache.hadoop.hbase.UnknownScannerException: Name:
> > -7220522571873774180
> >         at
> > org.apache.hadoop.hbase.HRegionServer.next(HRegionServer.java:1075)
> >         at
> > sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
> >         at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth
> > odAccessorImpl.java:25)
> >         at java.lang.reflect.Method.invoke(Method.java:585)
> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:379)
> >         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:596)
> >
> >         at org.apache.hadoop.ipc.Client.call(Client.java:482)
> >         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:184)
> >         at $Proxy1.next(Unknown Source)
> >         at
> > org.apache.hadoop.hbase.HTable$ClientScanner.next(HTable.java:831)
> >         ...
> >
> > there are totally 6 machines:
> > one running namenode, secondarynamenode, hbase master five
> > running datanode, regionserver
> >
> >
> > Cheers,
> > Cedric
> >
>

Mime
View raw message