hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yu Li <car...@gmail.com>
Subject Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Date Thu, 12 Jan 2017 02:29:24 GMT
bq. number_of_rows: 2147483647
>From the above line in your posted log, somehow your
hbase.client.scanner.caching
setting didn't take effect. Please make sure you modified the correct
hbase-site.xml on client side.

You could also set the caching in the application code, simply by invoking
Scan#setCaching

Best Regards,
Yu

On 11 January 2017 at 20:45, Rajeshkumar J <rajeshkumarit8292@gmail.com>
wrote:

> this is the log I am getting while running
>
> 2017-01-05 11:41:49,629 DEBUG
> [B.defaultRpcServer.handler=15,queue=0,port=16020] ipc.RpcServer:
> B.defaultRpcServer.handler=15,queue=0,port=16020: callId: 3 service:
> ClientService methodName: Scan size: 23 connection: hostname:port
> org.apache.hadoop.hbase.regionserver.LeaseException: lease '706' does not
> exist
> at org.apache.hadoop.hbase.regionserver.Leases.
> removeLease(Leases.java:221)
> at org.apache.hadoop.hbase.regionserver.Leases.
> cancelLease(Leases.java:206)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.
> scan(RSRpcServices.java:2491)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.
> callBlockingMethod(ClientProtos.java:32205)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:744)
> 2017-01-05 11:41:49,629 TRACE
> [B.defaultRpcServer.handler=18,queue=0,port=16020] ipc.RpcServer: callId:
> 2
> service: ClientService methodName: Scan size: 29 connection: hostname:port
> param: scanner_id: 706 number_of_rows: 2147483647 close_scanner: false
> next_call_seq: 0 client_handles_partials: true client_handles_heartbeats:
> true connection: hostname:port, response scanner_id: 706 more_results: true
> stale: false more_results_in_region: false queueTime: 1 processingTime:
> 60136 totalTime: 60137
>
> On Wed, Jan 11, 2017 at 4:45 PM, Rajeshkumar J <
> rajeshkumarit8292@gmail.com>
> wrote:
>
> > I have tried setting hbase.client.scanner.caching to 10, 100, 1000 and
> > 10000. But it still fails.
> >
> > On Wed, Jan 11, 2017 at 2:17 PM, Yu Li <carp84@gmail.com> wrote:
> >
> >> Hi Rajeshkumar,
> >>
> >> Simply speaking, in your case I'd suggest to set
> >> hbase.client.scanner.caching
> >> to a small value like 10 but not zero, and leave the
> >> hbase.client.scanner.caching or hbase.rpc.timeout setting as default.
> >> Below
> >> are some detailed explanation:
> >>
> >> Setting hbase.client.scanner.caching to zero will cause the scan returns
> >> no
> >> data (please check whether you are getting a wrong result with this
> >> setting), so please don't make such settings unless you intend to renew
> >> the
> >> scan lease. See RsRpcServices#scan source code for more details if
> >> interested.
> >>
> >> For version 1.1.1 (actually after HBASE-16973), the default value of
> >> hbase.client.scanner.caching
> >> is Integer.MAX_VALUE, and it won't return (for a single scanner.next
> call)
> >> until the result size reached hbase.client.scanner.max.result.size or
> >> times
> >> out. If you are using a sparse filter, then it's strongly suggested to
> >> set hbase.client.scanner.caching
> >> to some small value but not zero. See HBASE-16973 for more details if
> >> interested.
> >>
> >> Regarding hbase.rpc.timeout and hbase.client.scanner.timeout.period:
> >> * For scan it's hbase.client.scanner.timeout.period taking effect
> >> * For all other kinds of non-admin operations like
> >> get/put/append/increment/multi hbase.rpc.timeout takes effect, unless
> you
> >> specified hbase.client.operation.timeout.
> >> Check ClientScanner#scannerTimeout, ScannerCallable#call,
> >> AbstractRpcClient#channelOperationTimeout and
> >> AbstractRpcClient#callBlockingMethod for more details if interested.
> >>
> >> Best Regards,
> >> Yu
> >>
> >> On 11 January 2017 at 15:29, Rajeshkumar J <rajeshkumarit8292@gmail.com
> >
> >> wrote:
> >>
> >> > Hi, As i got lease expire exception  i have increased the values of
> >> these
> >> > properties hbase.client.scanner.timeout.period and hbase.rpc.timeout.
> >> Is
> >> > there any disadvantage increasing these two values?
> >> >
> >> > On Mon, Jan 2, 2017 at 11:01 AM, Rajeshkumar J <
> >> > rajeshkumarit8292@gmail.com>
> >> > wrote:
> >> >
> >> > > I am using hbase 1.1.1.
> >> > >
> >> > >   I have following property value in hbase-site.xml and I didn't set
> >> > value
> >> > > hbase.client.scanner.caching property(so it must pick default value
> >> for
> >> > > this)
> >> > >
> >> > >         <property>
> >> > >         <name>hbase.client.scanner.timeout.period</name>
> >> > >         <value>60000</value>
> >> > >         </property>
> >> > >
> >> > >         <property>
> >> > >         <name>hbase.rpc.timeout</name>
> >> > >         <value>60000</value>
> >> > >         </property>
> >> > > When scanning 30 tables in hbase(each table holds 45 million
> records)
> >> I
> >> > > got the following error
> >> > >
> >> > > WARN client.ScannerCallable [main]: Ignore, probably already closed
> >> > > org.apache.hadoop.hbase.regionserver.LeaseException:
> >> > > org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
> >> > exist
> >> > >
> >> > > But when I added hbase.client.scanner.caching property value as
> zero I
> >> > > didn't get any error. Why this happened?
> >> > > Even though I gave zero it supposed to pick the default value? How
> can
> >> > > this process succeeds?
> >> > >
> >> > > Thanks
> >> > >
> >> > > On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <elserj@apache.org>
> >> wrote:
> >> > >
> >> > >> I am having an extremely difficult time parsing this one,
> >> Rajeshkumar.
> >> > >>
> >> > >> If you still have a question/something you don't understand, I'd
> >> > >> appreciate it if you could try to re-state it a little more
> clearly.
> >> > >>
> >> > >>
> >> > >> Rajeshkumar J wrote:
> >> > >>
> >> > >>> I am using hbase 1.1.1. Yes while setting zero it must pick
the
> >> default
> >> > >>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we
have
> set
> >> the
> >> > >>> value as INTEGER.MAX_VALUE only but earlier it had failed
but now
> >> > >>> succeeded. So only I am asking this. Also I didn't change
any
> >> property
> >> > >>> whatever I used in process which failed in the process which
> >> succeeded
> >> > >>> except assigning Zero value for this property
> >> > >>>
> >> > >>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<elserj@apache.org>
> >> wrote:
> >> > >>>
> >> > >>> Most likely, since you gave a nonsensical value, HBase used
a
> >> default
> >> > >>>> value instead of the one you provided. Since you have
not shared
> >> the
> >> > >>>> version of HBase which you are using, I would recommend
that you
> >> look
> >> > at
> >> > >>>> the code. It should be very obvious what value is being
used
> >> instead
> >> > of
> >> > >>>> the
> >> > >>>> bogus value you provided.
> >> > >>>>
> >> > >>>>
> >> > >>>> Rajeshkumar J wrote:
> >> > >>>>
> >> > >>>> I have tried setting hbase.client.scanner.caching property
to 0
> >> with
> >> > >>>>> above
> >> > >>>>> two values as 60000. It runs fine. But
> >> hbase.client.scanner.caching
> >> > >>>>> property tells no of rows to be fetched for every
scanner but
> >> here I
> >> > >>>>> have
> >> > >>>>> given zero. How this worked?
> >> > >>>>>
> >> > >>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<elserj@apache.org>
> >> > >>>>>  wrote:
> >> > >>>>>
> >> > >>>>> hbase.client.scanner.timeout.period is a timeout specifically
> for
> >> > RPCs
> >> > >>>>>
> >> > >>>>>> that come from the HBase Scanner classes (e.g.
ClientScanner)
> >> while
> >> > >>>>>> hbase.rpc.timeout is the default timeout for any
RPC. I believe
> >> that
> >> > >>>>>> the
> >> > >>>>>> hbase.client.scanner.timeout.period is also used
by the
> >> > >>>>>> RegionServers to
> >> > >>>>>> define the lifetime of the Lease (the cause of
the
> LeaseException
> >> > >>>>>> you're
> >> > >>>>>> seeing).
> >> > >>>>>>
> >> > >>>>>> Generally, when you see these kinds of exceptions
while
> scanning
> >> > data
> >> > >>>>>> in
> >> > >>>>>> HBase, it is just a factor of your hardware and
current
> >> performance
> >> > >>>>>> (in
> >> > >>>>>> other words, how long it takes to read your data).
I can't
> really
> >> > >>>>>> give a
> >> > >>>>>> firm answer because it is dependent on your system's
> performance.
> >> > You
> >> > >>>>>> should be able to approximate the performance
with some
> >> > >>>>>> back-of-the-envelope math.
> >> > >>>>>>
> >> > >>>>>> Some changes like https://issues.apache.org/jira
> >> /browse/HBASE-13090
> >> > >>>>>> and
> >> > >>>>>> https://issues.apache.org/jira/browse/HBASE-13333
should help
> >> > reduce
> >> > >>>>>> the
> >> > >>>>>> need for you to tweak configuration properties
in the future.
> >> > >>>>>>
> >> > >>>>>>
> >> > >>>>>> Rajeshkumar J wrote:
> >> > >>>>>>
> >> > >>>>>> Hi,
> >> > >>>>>>
> >> > >>>>>>>         I have following property value as
below
> >> > >>>>>>>
> >> > >>>>>>>            <property>
> >> > >>>>>>>            <name>hbase.client.scanner.timeout.period</name>
> >> > >>>>>>>            <value>60000</value>
> >> > >>>>>>>            </property>
> >> > >>>>>>>
> >> > >>>>>>>            <property>
> >> > >>>>>>>            <name>hbase.rpc.timeout</name>
> >> > >>>>>>>            <value>60000</value>
> >> > >>>>>>>            </property>
> >> > >>>>>>> When scanning 30 tables in hbase(each table
holds 45 million
> >> > >>>>>>> records) I
> >> > >>>>>>> got
> >> > >>>>>>>
> >> > >>>>>>> WARN client.ScannerCallable [main]: Ignore,
probably already
> >> closed
> >> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
> >> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
lease
> does
> >> > not
> >> > >>>>>>> exist
> >> > >>>>>>>
> >> > >>>>>>> But when I change the value to
> >> > >>>>>>>
> >> > >>>>>>>           <property>
> >> > >>>>>>>            <name>hbase.client.scanner.timeout.period</name>
> >> > >>>>>>>            <value>70000</value>
> >> > >>>>>>>            </property>
> >> > >>>>>>>
> >> > >>>>>>>            <property>
> >> > >>>>>>>            <name>hbase.rpc.timeout</name>
> >> > >>>>>>>            <value>70000</value>
> >> > >>>>>>>            </property>
> >> > >>>>>>> It runs successfully.
> >> > >>>>>>> Can any one tell me the reason for this failure
and also is
> >> there
> >> > any
> >> > >>>>>>> factor to calculate these property values
for any data.
> >> > >>>>>>>
> >> > >>>>>>>
> >> > >>>>>>>
> >> > >>>>>>>
> >> > >>>
> >> > >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message