hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillaume Perrot <gper...@ubikod.com>
Subject Re: Differences between ScannerTimeoutException and LeaseException during a scanner
Date Thu, 15 Nov 2012 18:12:34 GMT
No, but we'll try that soon.

2012/11/15 Ted Yu <yuzhihong@gmail.com>

> Thanks for sharing your experience.
>
> Have you set hbase.rpc.timeout > hbase.regionserver.lease.period in your
> environment ?
>
> Cheers
>
> On Thu, Nov 15, 2012 at 7:45 AM, Guillaume Perrot <gperrot@capptain.com
> >wrote:
>
> > It looks like a "race condition" between the check that the lease is
> > expired in HRegionServer.next, and the LeaseListener.leaseExpired
> methods.
> > If the next call from the client is done before the lease listener do its
> > job, the scanner still exists in the map but is expired, which causes
> > LeaseException instead of UnknownScannerException.
> > Indeed I found this in the logs:
> >
> > 2012-11-15 08:45:34,003 ERROR
> > org.apache.hadoop.hbase.regionserver.HRegionServer:
> > org.apache.hadoop.hbase.regionserver.LeaseException: lease
> > '4939065839430667309' does not exist
> >  at
> > org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:231)
> > at
> >
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2117)
> >  at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >  at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> >
> >
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> >  at
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> >
> > Then
> >
> > 2012-11-15 08:48:37,734 INFO
> > org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> > 4939065839430667309 lease expired
> >
> > The scanner is properly closed way too late!
> > I just realized there is already a related issue about this:
> > https://issues.apache.org/jira/browse/HBASE-6856
> >
> > 2012/11/15 Guillaume Perrot <gperrot@ubikod.com>
> >
> > > Hi,
> > > Similar questions have been mailed in the past, but they are not clear
> > > enough to me, so here is a new one:
> > > In HBase 0.90.3, when a client scanner was too slow to advance (no
> actual
> > > next call within 60s), we had a nice ScannerTimeoutException in client
> > > (UnknownScannerException in region server).
> > > With HBase 0.92.1, when I simulate such a slow scanner I still have
> this
> > > exception.
> > >
> > > But in our production environment, scanners that used to
> > > produce ScannerTimeoutException in HBase 0.90.3 no longer produce them
> > > since we upgraded to HBase 0.92.1.
> > > Instead we now have:
> > > org.apache.hadoop.hbase.regionserver.LeaseException:
> > > org.apache.hadoop.hbase.regionserver.LeaseException: lease
> > > '2770824890045864909' does not exist
> > > at
> > org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:231)
> > >  at
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2117)
> > > at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> > >  at
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > > at java.lang.reflect.Method.invoke(Method.java:597)
> > >  at
> > >
> >
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> > > at
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> > >
> > > I don't uderstand the workflow leading to this exception instead of a
> > > ScannerTimeoutException, any local attempt to reproduce this problem
> > leads
> > > to a ScannerTimeoutException and not a LeaseException.
> > > Any idea ?
> > > Thanks in advance.
> > >
> >
> >
> >
> > --
> >   *Guillaume Perrot
> > **Software Engineer
> > *[image: logo]
> >    *Contact info *
> > gperrot@capptain.com
> > www.capptain.com
> >
> >  *Rennes Office*
> > Office: +33 2 99 65 69 13
> > 10 rue Jean-Marie Duhamel
> > 35000 Rennes
> > France *Paris Office*
> > Office: +33 1 84 06 13 85
> > Fax: +33 9 57 72 20 18
> > 18 rue Tronchet
> > 75008 Paris
> > France
> >
> > IMPORTANT NOTICE – UBIKOD and CAPPTAIN are registered trademarks of
> UBIKOD
> > S.A.R.L., all copyrights are reserved. The contents of this email and
> > attachments are confidential and may be subject to legal privilege and/or
> > protected by copyright. Copying or communicating any part of it to others
> > is prohibited and may be unlawful. If you are not the intended recipient
> > you must not use, copy, distribute or rely on this email and should
> please
> > return it immediately or notify us by telephone. At present the integrity
> > of email across the Internet cannot be guaranteed. Therefore UBIKOD
> > S.A.R.L. will not accept liability for any claims arising as a result of
> > the use of this medium for transmissions by or to UBIKOD S.A.R.L.. UBIKOD
> > S.A.R.L. may exercise any of its rights under relevant law, to monitor
> the
> > content of all electronic communications. You should therefore be aware
> > that this communication and any responses might have been monitored, and
> > may be accessed by UBIKOD S.A.R.L. The views expressed in this document
> are
> > that of the individual and may not necessarily constitute or imply its
> > endorsement or recommendation by UBIKOD S.A.R.L. The content of this
> > electronic mail may be subject to the confidentiality terms of a
> > "Non-Disclosure Agreement" (NDA).
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message