hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wht (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15914) lease expired exception when do scan with multiple thread
Date Mon, 30 May 2016 03:31:12 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

wht updated HBASE-15914:
------------------------
    Description: 
hello, i am a freshman with Hbase ,now i do the performace test with hbase in our project.
i use scan api to scan a table with filters. because we need the good performance ,so we scan
each region with a single thread , total 24 regions, so we start 24 scan task to scan . each
scan task we get a scanner to scan .

but now i have a exception below: 

2016-05-30 11:18:46,399 | DEBUG | B.defaultRpcServer.handler=46,queue=6,port=21302 | B.defaultRpcServer.handler=46,queue=6,port=21302:
callId: 102 service: ClientService methodName: Scan size: 22 connection: XXXXXXXXXX:59645
| org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
org.apache.hadoop.hbase.regionserver.LeaseException: lease '11' does not exist
        at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
        at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)
2016-05-30 11:18:46,417 | DEBUG | B.defaultRpcServer.handler=29,queue=9,port=21302 | B.defaultRpcServer.handler=29,queue=9,port=21302:
callId: 104 service: ClientService methodName: Scan size: 22 connection: xxxxxxxx:59645 |
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
org.apache.hadoop.hbase.regionserver.LeaseException: lease '17' does not exist
        at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
        at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)
2016-05-30 11:18:46,427 | DEBUG | B.defaultRpcServer.handler=72,queue=12,port=21302 | B.defaultRpcServer.handler=72,queue=12,port=21302:
callId: 94 service: ClientService methodName: Scan size: 22 connection: XXXXXXXX:59645 | org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
org.apache.hadoop.hbase.regionserver.LeaseException: lease '10' does not exist
        at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
        at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)
2016-05-30 11:18:46,454 | DEBUG | B.defaultRpcServer.handler=59,queue=19,port=21302 | B.defaultRpcServer.handler=59,queue=19,port=21302:
callId: 88 service: ClientService methodName: Scan size: 22 connection: xxxxxxxxx:59645 |
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
org.apache.hadoop.hbase.regionserver.LeaseException: lease '15' does not exist
        at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
        at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)


can any one give me some help?

some information about my hbase cluster:

hbase version: 1.1.4

Configure: 
hbase.client.scanner.timeout.period    120s
hbase.rpc.timeout      120s

setCaching(1000)


  was:
hello, i am a freshman with Hbase ,now i do the performace test with hbase in our project.
i use scan api to scan a table with filters. because we need the good performance ,so we scan
each region with a single thread , total 24 regions, so we start 24 scan task to scan . each
scan task we get a scanner to scan .

but now i have a exception below: 

2016-05-30 11:18:46,399 | DEBUG | B.defaultRpcServer.handler=46,queue=6,port=21302 | B.defaultRpcServer.handler=46,queue=6,port=21302:
callId: 102 service: ClientService methodName: Scan size: 22 connection: 10.146.228.155:59645
| org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
org.apache.hadoop.hbase.regionserver.LeaseException: lease '11' does not exist
        at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
        at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)
2016-05-30 11:18:46,417 | DEBUG | B.defaultRpcServer.handler=29,queue=9,port=21302 | B.defaultRpcServer.handler=29,queue=9,port=21302:
callId: 104 service: ClientService methodName: Scan size: 22 connection: 10.146.228.155:59645
| org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
org.apache.hadoop.hbase.regionserver.LeaseException: lease '17' does not exist
        at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
        at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)
2016-05-30 11:18:46,427 | DEBUG | B.defaultRpcServer.handler=72,queue=12,port=21302 | B.defaultRpcServer.handler=72,queue=12,port=21302:
callId: 94 service: ClientService methodName: Scan size: 22 connection: 10.146.228.155:59645
| org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
org.apache.hadoop.hbase.regionserver.LeaseException: lease '10' does not exist
        at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
        at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)
2016-05-30 11:18:46,454 | DEBUG | B.defaultRpcServer.handler=59,queue=19,port=21302 | B.defaultRpcServer.handler=59,queue=19,port=21302:
callId: 88 service: ClientService methodName: Scan size: 22 connection: 10.146.228.155:59645
| org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
org.apache.hadoop.hbase.regionserver.LeaseException: lease '15' does not exist
        at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
        at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)


can any one give me some help?

some information about my hbase cluster:

hbase version: 1.1.4

Configure: 
hbase.client.scanner.timeout.period    120s
hbase.rpc.timeout      120s

setCaching(1000)



> lease expired exception when do scan with multiple thread
> ---------------------------------------------------------
>
>                 Key: HBASE-15914
>                 URL: https://issues.apache.org/jira/browse/HBASE-15914
>             Project: HBase
>          Issue Type: Wish
>          Components: regionserver
>            Reporter: wht
>            Priority: Minor
>
> hello, i am a freshman with Hbase ,now i do the performace test with hbase in our project.
i use scan api to scan a table with filters. because we need the good performance ,so we scan
each region with a single thread , total 24 regions, so we start 24 scan task to scan . each
scan task we get a scanner to scan .
> but now i have a exception below: 
> 2016-05-30 11:18:46,399 | DEBUG | B.defaultRpcServer.handler=46,queue=6,port=21302 |
B.defaultRpcServer.handler=46,queue=6,port=21302: callId: 102 service: ClientService methodName:
Scan size: 22 connection: XXXXXXXXXX:59645 | org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
> org.apache.hadoop.hbase.regionserver.LeaseException: lease '11' does not exist
>         at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
>         at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
>         at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
>         at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
>         at java.lang.Thread.run(Thread.java:745)
> 2016-05-30 11:18:46,417 | DEBUG | B.defaultRpcServer.handler=29,queue=9,port=21302 |
B.defaultRpcServer.handler=29,queue=9,port=21302: callId: 104 service: ClientService methodName:
Scan size: 22 connection: xxxxxxxx:59645 | org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
> org.apache.hadoop.hbase.regionserver.LeaseException: lease '17' does not exist
>         at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
>         at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
>         at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
>         at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
>         at java.lang.Thread.run(Thread.java:745)
> 2016-05-30 11:18:46,427 | DEBUG | B.defaultRpcServer.handler=72,queue=12,port=21302 |
B.defaultRpcServer.handler=72,queue=12,port=21302: callId: 94 service: ClientService methodName:
Scan size: 22 connection: XXXXXXXX:59645 | org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
> org.apache.hadoop.hbase.regionserver.LeaseException: lease '10' does not exist
>         at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
>         at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
>         at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
>         at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
>         at java.lang.Thread.run(Thread.java:745)
> 2016-05-30 11:18:46,454 | DEBUG | B.defaultRpcServer.handler=59,queue=19,port=21302 |
B.defaultRpcServer.handler=59,queue=19,port=21302: callId: 88 service: ClientService methodName:
Scan size: 22 connection: xxxxxxxxx:59645 | org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
> org.apache.hadoop.hbase.regionserver.LeaseException: lease '15' does not exist
>         at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
>         at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
>         at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2551)
>         at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2134)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:103)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
>         at java.lang.Thread.run(Thread.java:745)
> can any one give me some help?
> some information about my hbase cluster:
> hbase version: 1.1.4
> Configure: 
> hbase.client.scanner.timeout.period    120s
> hbase.rpc.timeout      120s
> setCaching(1000)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message