phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kiran Kumar Maturi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-5267) With namespaces enabled Phoenix client times out with high loads
Date Thu, 02 May 2019 14:49:00 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kiran Kumar Maturi updated PHOENIX-5267:
----------------------------------------
    Description: 
Steps to reproduce:
 * Enable namespaces for phoenix 4.14.1 and hbase 1.3
 * Run high load using pherf client with 48 threads

After sometime the client hangs. and gives timeout exception
{code:java}
[pool-1-thread-1] WARN org.apache.phoenix.pherf.workload.WriteWorkload -

java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException:
callTimeout=1200000, callDuration=1238263: Call to <host> failed on local exception:
org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=857, waitTime=120001, operationTimeout=120000
expired. row '^@TEST^@TABLE' on table 'SYSTEM:CATALOG' at region=SYSTEM:CATALOG,1556024429507.0f80d6de0a002d1421b8fd384e956254.,
hostname=<host>, seqNum=2

    at java.util.concurrent.FutureTask.report(FutureTask.java:122)

    at java.util.concurrent.FutureTask.get(FutureTask.java:192)

    at org.apache.phoenix.pherf.workload.WriteWorkload.waitForBatches(WriteWorkload.java:239)

    at org.apache.phoenix.pherf.workload.WriteWorkload.exec(WriteWorkload.java:189)

    at org.apache.phoenix.pherf.workload.WriteWorkload.access$100(WriteWorkload.java:56)

    at org.apache.phoenix.pherf.workload.WriteWorkload$1.run(WriteWorkload.java:165)

    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

    at java.util.concurrent.FutureTask.run(FutureTask.java:266)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

    at java.lang.Thread.run(Thread.java:748)

Caused by: org.apache.phoenix.exception.PhoenixIOException: callTimeout=1200000, callDuration=1238263:
Call to <host> failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException:
Call id=857, waitTime=120001, operationTimeout=120000 expired. row '^@TEST^@TABLE' on table
'SYSTEM:CATALOG' at region=SYSTEM:CATALOG,1556024429507.0f80d6de0a002d1421b8fd384e956254.,
hostname=<host>, seqNum=2

    at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)

    at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:1379)

    at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:1343)

    at org.apache.phoenix.query.ConnectionQueryServicesImpl.getTable(ConnectionQueryServicesImpl.java:1560)

    at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:644)

    at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:538)

    at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:530)

    at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:526)

    at org.apache.phoenix.execute.MutationState.validateAndGetServerTimestamp(MutationState.java:755)

    at org.apache.phoenix.execute.MutationState.validateAll(MutationState.java:743)

    at org.apache.phoenix.execute.MutationState.send(MutationState.java:875)

    at org.apache.phoenix.execute.MutationState.send(MutationState.java:1360)

    at org.apache.phoenix.execute.MutationState.commit(MutationState.java:1183)

    at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:670)

    at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:666)

    at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)

    at org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:666)

    at org.apache.phoenix.pherf.workload.WriteWorkload$2.call(WriteWorkload.java:297)

    at org.apache.phoenix.pherf.workload.WriteWorkload$2.call(WriteWorkload.java:256)


{code}

  was:
Steps to reproduce:
 * Enable namespaces for phoenix 4.14.1 and hbase 1.3
 * Run high load using pherf client with 48 threads

After sometime the client hangs. and gives timeout exception
{code:java}
[pool-1-thread-1] WARN org.apache.phoenix.pherf.workload.WriteWorkload -

java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException:
callTimeout=1200000, callDuration=1238263: Call to <host> failed on local exception:
org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=857, waitTime=120001, operationTimeout=120000
expired. row '^@test^@table' on table 'SYSTEM:CATALOG' at region=SYSTEM:CATALOG,1556024429507.0f80d6de0a002d1421b8fd384e956254.,
hostname=<host>, seqNum=2

    at java.util.concurrent.FutureTask.report(FutureTask.java:122)

    at java.util.concurrent.FutureTask.get(FutureTask.java:192)

    at org.apache.phoenix.pherf.workload.WriteWorkload.waitForBatches(WriteWorkload.java:239)

    at org.apache.phoenix.pherf.workload.WriteWorkload.exec(WriteWorkload.java:189)

    at org.apache.phoenix.pherf.workload.WriteWorkload.access$100(WriteWorkload.java:56)

    at org.apache.phoenix.pherf.workload.WriteWorkload$1.run(WriteWorkload.java:165)

    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

    at java.util.concurrent.FutureTask.run(FutureTask.java:266)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

    at java.lang.Thread.run(Thread.java:748)

Caused by: org.apache.phoenix.exception.PhoenixIOException: callTimeout=1200000, callDuration=1238263:
Call to <host> failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException:
Call id=857, waitTime=120001, operationTimeout=120000 expired. row '^@TEST^@TABLE' on table
'SYSTEM:CATALOG' at region=SYSTEM:CATALOG,1556024429507.0f80d6de0a002d1421b8fd384e956254.,
hostname=<host>, seqNum=2

    at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)

    at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:1379)

    at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:1343)

    at org.apache.phoenix.query.ConnectionQueryServicesImpl.getTable(ConnectionQueryServicesImpl.java:1560)

    at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:644)

    at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:538)

    at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:530)

    at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:526)

    at org.apache.phoenix.execute.MutationState.validateAndGetServerTimestamp(MutationState.java:755)

    at org.apache.phoenix.execute.MutationState.validateAll(MutationState.java:743)

    at org.apache.phoenix.execute.MutationState.send(MutationState.java:875)

    at org.apache.phoenix.execute.MutationState.send(MutationState.java:1360)

    at org.apache.phoenix.execute.MutationState.commit(MutationState.java:1183)

    at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:670)

    at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:666)

    at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)

    at org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:666)

    at org.apache.phoenix.pherf.workload.WriteWorkload$2.call(WriteWorkload.java:297)

    at org.apache.phoenix.pherf.workload.WriteWorkload$2.call(WriteWorkload.java:256)


{code}


> With namespaces enabled Phoenix client times out with high loads
> ----------------------------------------------------------------
>
>                 Key: PHOENIX-5267
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5267
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.14.1
>            Reporter: Kiran Kumar Maturi
>            Priority: Major
>
> Steps to reproduce:
>  * Enable namespaces for phoenix 4.14.1 and hbase 1.3
>  * Run high load using pherf client with 48 threads
> After sometime the client hangs. and gives timeout exception
> {code:java}
> [pool-1-thread-1] WARN org.apache.phoenix.pherf.workload.WriteWorkload -
> java.util.concurrent.ExecutionException: org.apache.phoenix.exception.PhoenixIOException:
callTimeout=1200000, callDuration=1238263: Call to <host> failed on local exception:
org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=857, waitTime=120001, operationTimeout=120000
expired. row '^@TEST^@TABLE' on table 'SYSTEM:CATALOG' at region=SYSTEM:CATALOG,1556024429507.0f80d6de0a002d1421b8fd384e956254.,
hostname=<host>, seqNum=2
>     at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>     at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>     at org.apache.phoenix.pherf.workload.WriteWorkload.waitForBatches(WriteWorkload.java:239)
>     at org.apache.phoenix.pherf.workload.WriteWorkload.exec(WriteWorkload.java:189)
>     at org.apache.phoenix.pherf.workload.WriteWorkload.access$100(WriteWorkload.java:56)
>     at org.apache.phoenix.pherf.workload.WriteWorkload$1.run(WriteWorkload.java:165)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>     at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.phoenix.exception.PhoenixIOException: callTimeout=1200000, callDuration=1238263:
Call to <host> failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException:
Call id=857, waitTime=120001, operationTimeout=120000 expired. row '^@TEST^@TABLE' on table
'SYSTEM:CATALOG' at region=SYSTEM:CATALOG,1556024429507.0f80d6de0a002d1421b8fd384e956254.,
hostname=<host>, seqNum=2
>     at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:144)
>     at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:1379)
>     at org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:1343)
>     at org.apache.phoenix.query.ConnectionQueryServicesImpl.getTable(ConnectionQueryServicesImpl.java:1560)
>     at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:644)
>     at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:538)
>     at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:530)
>     at org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:526)
>     at org.apache.phoenix.execute.MutationState.validateAndGetServerTimestamp(MutationState.java:755)
>     at org.apache.phoenix.execute.MutationState.validateAll(MutationState.java:743)
>     at org.apache.phoenix.execute.MutationState.send(MutationState.java:875)
>     at org.apache.phoenix.execute.MutationState.send(MutationState.java:1360)
>     at org.apache.phoenix.execute.MutationState.commit(MutationState.java:1183)
>     at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:670)
>     at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:666)
>     at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>     at org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:666)
>     at org.apache.phoenix.pherf.workload.WriteWorkload$2.call(WriteWorkload.java:297)
>     at org.apache.phoenix.pherf.workload.WriteWorkload$2.call(WriteWorkload.java:256)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message