hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajeshkumar J <rajeshkumarit8...@gmail.com>
Subject Re: Lease exception
Date Tue, 27 Dec 2016 05:20:55 GMT
sorry for the delay. I didn't get the lease concept here whether it is
specific to hbase or like lease in hadoop?

On Thu, Dec 22, 2016 at 3:13 PM, Phil Yang <ud1937@gmail.com> wrote:

> https://github.com/apache/hbase/blob/rel/1.1.1/hbase-
> server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.
> java#L2491
>
> There is a TTL for scanners at server, to prevent client don't close the
> scanners and they leak. The TTL is configured by
> hbase.client.scanner.timeout.period at server and refreshed when a scan
> RPC
> request comes . The TTLs of all scanners are managed by Lease. Your error
> happens when server closes a scanner but in Lease it is already expired. So
> I think you can try to increase  hbase.client.scanner.timeout.period at
> server or decrease your hbase.client.scanner.timeout.period at client to
> prevent the scanner expired before its scanning done.
> hbase.client.scanner.timeout.period is used both at client and server, may
> be different if you change one of sides.
>
> BTW, I still suggest that you can upgrade your cluster and client. 1.1.1
> has some data-loss bugs on scanning.
>
> Thanks,
> Phil
>
>
> 2016-12-22 17:26 GMT+08:00 Rajeshkumar J <rajeshkumarit8292@gmail.com>:
>
> > can you please explain what is the cause of this lease exception and is
> > there any solve this in current version
> >
> > Thanks
> >
> > On Thu, Dec 22, 2016 at 2:54 PM, Phil Yang <ud1937@gmail.com> wrote:
> >
> > > In fact at client the rpc timeout of scan request is also
> > > hbase.client.scanner.timeout.period which replaces the
> > > deprecated hbase.regionserver.lease.period.
> > >
> > > Your code that throws LeaseException has been removed by HBASE-16604,
> > maybe
> > > you can try to upgrade your cluster to 1.1.7? Your client can also
> > upgrade
> > > to 1.1.7 which will ignore UnknowScannerException and retry when the
> > lease
> > > is expired at server.
> > >
> > > Thanks,
> > > Phil
> > >
> > >
> > > 2016-12-22 16:51 GMT+08:00 Rajeshkumar J <rajeshkumarit8292@gmail.com
> >:
> > >
> > > > Also there is a solution what i have found from hbase user guide that
> > > > hbase.rpc.timeout must be greater than hbase.client.scanner.timeout.
> > > > period.
> > > > How these two properties plays a part in the above exception. Please
> > can
> > > > anyone explain?
> > > >
> > > > On Wed, Dec 21, 2016 at 9:39 PM, Rajeshkumar J <
> > > > rajeshkumarit8292@gmail.com>
> > > > wrote:
> > > >
> > > > > I am using hbase version 1.1.1
> > > > > Also I didn't understand something here. Whenever a scanner.next()
> is
> > > > > called it needs to return rows(based on caching value) within
> leasing
> > > > > period or else scanner client will be closed eventually throwing
> this
> > > > > exception. Correct me as I didn't get the clear understanding of
> this
> > > > issue
> > > > >
> > > > > On Wed, Dec 21, 2016 at 7:31 PM, Ted Yu <yuzhihong@gmail.com>
> wrote:
> > > > >
> > > > >> Which hbase release are you using ?
> > > > >>
> > > > >> There is heartbeat support when scanning.
> > > > >> Looks like the version you use doesn't have this support.
> > > > >>
> > > > >> Cheers
> > > > >>
> > > > >> > On Dec 21, 2016, at 4:02 AM, Rajeshkumar J <
> > > > rajeshkumarit8292@gmail.com>
> > > > >> wrote:
> > > > >> >
> > > > >> > Hi,
> > > > >> >
> > > > >> >   Thanks for the reply. I have properties as below
> > > > >> >
> > > > >> > <property>
> > > > >> >    <name>hbase.regionserver.lease.period</name>
> > > > >> >    <value>900000</value>
> > > > >> >  </property>
> > > > >> >  <property>
> > > > >> >    <name>hbase.rpc.timeout</name>
> > > > >> >    <value>900000>/value>
> > > > >> >  </property>
> > > > >> >
> > > > >> >
> > > > >> > Correct me If I am wrong.
> > > > >> >
> > > > >> > I know hbase.regionserver.lease.period, which says how long
a
> > > scanner
> > > > >> > lives between calls to scanner.next().
> > > > >> >
> > > > >> > As far as I understand when scanner.next() is called it
will
> fetch
> > > no
> > > > >> > of rows as in *hbase.client.scanner.caching. *When this
fetching
> > > > >> > process takes more than lease period it will close the scanner
> > > object.
> > > > >> > so this exception occuring?
> > > > >> >
> > > > >> >
> > > > >> > Thanks,
> > > > >> >
> > > > >> > Rajeshkumar J
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Wed, Dec 21, 2016 at 5:07 PM, Richard Startin <
> > > > >> richardstartin@outlook.com
> > > > >> >> wrote:
> > > > >> >
> > > > >> >> It means your lease on a region server has expired during
a
> call
> > to
> > > > >> >> resultscanner.next(). This happens on a slow call to
next().
> You
> > > can
> > > > >> either
> > > > >> >> embrace it or "fix" it by making sure hbase.rpc.timeout
exceeds
> > > > >> >> hbase.regionserver.lease.period.
> > > > >> >>
> > > > >> >> https://richardstartin.com
> > > > >> >>
> > > > >> >> On 21 Dec 2016, at 11:30, Rajeshkumar J <
> > > rajeshkumarit8292@gmail.com
> > > > <
> > > > >> >> mailto:rajeshkumarit8292@gmail.com>> wrote:
> > > > >> >>
> > > > >> >> Hi,
> > > > >> >>
> > > > >> >>  I have faced below issue in our production cluster
> > > > >> >>
> > > > >> >> org.apache.hadoop.hbase.regionserver.LeaseException:
> > > > >> >> org.apache.hadoop.hbase.regionserver.LeaseException:
lease
> > > '166881'
> > > > >> does
> > > > >> >> not exist
> > > > >> >> at org.apache.hadoop.hbase.regionserver.Leases.
> > > > >> >> removeLease(Leases.java:221)
> > > > >> >> at org.apache.hadoop.hbase.regionserver.Leases.
> > > > >> >> cancelLease(Leases.java:206)
> > > > >> >> at
> > > > >> >> org.apache.hadoop.hbase.regionserver.RSRpcServices.
> > > > >> >> scan(RSRpcServices.java:2491)
> > > > >> >> at
> > > > >> >> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$
> > > > >> ClientService$2.
> > > > >> >> callBlockingMethod(ClientProtos.java:32205)
> > > > >> >> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:
> > 2114)
> > > > >> >> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.
> > java:101)
> > > > >> >> at
> > > > >> >> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExec
> > > > >> utor.java:130)
> > > > >> >> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.
> > > > java:107)
> > > > >> >> at java.lang.Thread.run(Thread.java:744)
> > > > >> >>
> > > > >> >>
> > > > >> >> Can any one explain what is lease exception
> > > > >> >>
> > > > >> >> Thanks,
> > > > >> >> Rajeshkumar J
> > > > >> >>
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message