phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PHOENIX-5007) DELETE FROM times out - even with large HBase timeout setting
Date Thu, 08 Nov 2018 00:15:00 GMT
Lars Hofhansl created PHOENIX-5007:
--------------------------------------

             Summary: DELETE FROM times out - even with large HBase timeout setting
                 Key: PHOENIX-5007
                 URL: https://issues.apache.org/jira/browse/PHOENIX-5007
             Project: Phoenix
          Issue Type: Bug
            Reporter: Lars Hofhansl


It seems there is currently no way to make a somewhat large DELETE FROM work without a timeout.
"Large" here includes size equivalent to a single HBase region, which means we cannot write
a M/R (or Spark) task to delete a lot of data.

This is the exception:
{code:java}
0: jdbc:phoenix:localhost> delete from test where v1 < 0.999;
Error: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311:
Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException:
Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5.,
hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099 (state=08000,code=101)
org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException:
Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311:
Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException:
Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5.,
hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099

at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1296)
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1237)
at org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
at org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
at org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
at org.apache.phoenix.iterate.BaseGroupedAggregatingResultIterator.next(BaseGroupedAggregatingResultIterator.java:64)
at org.apache.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:39)
at org.apache.phoenix.compile.DeleteCompiler$ServerSelectDeleteMutationPlan.execute(DeleteCompiler.java:791)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:408)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:391)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:390)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:378)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1825)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:813)
at sqlline.SqlLine.begin(SqlLine.java:686)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:291)
Caused by: java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException:
Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311:
Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException:
Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5.,
hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099

at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:1291)
... 19 more
Caused by: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311:
Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException:
Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5.,
hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099

at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
at org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:171)
at org.apache.phoenix.iterate.LookAheadResultIterator$1.advance(LookAheadResultIterator.java:47)
at org.apache.phoenix.iterate.LookAheadResultIterator.init(LookAheadResultIterator.java:59)
at org.apache.phoenix.iterate.LookAheadResultIterator.peek(LookAheadResultIterator.java:73)
at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:135)
at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:120)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask.run(JobManager.java:183)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311:
Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException:
Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5.,
hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099

at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
at org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:156)
at org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:164)
... 10 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36,
exceptions:
Fri Oct 26 15:36:07 PDT 2018, null, java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311:
Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException:
Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5.,
hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099

at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:329)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:242)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:58)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:219)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:277)
at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:438)
at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:312)
at org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:147)
... 11 more
Caused by: java.net.SocketTimeoutException: callTimeout=180000, callDuration=180311: Call
to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException:
Call id=804, waitTime=59689, rpcTimetout=59678 row '�^�1' on table 'TEST' at region=TEST,\x82\x5CX-,1540590045706.b603f0cba6793295681c8ac18d93b1e5.,
hostname=lhofhansl-wsl2,16201,1540404153161, seqNum=211099
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:178)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
... 3 more
Caused by: java.io.IOException: Call to lhofhansl-wsl2/127.0.1.1:16201 failed on local exception:
org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689, rpcTimetout=59678
at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:180)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:389)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:94)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:409)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:405)
at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:188)
at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:625)
at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:700)
at org.apache.phoenix.shaded.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:428)
... 1 more
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=804, waitTime=59689,
rpcTimetout=59678
... 5 more
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message