cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rabii lamriq <lam...@gmail.com>
Subject Unexpected error during query + Operation timed out
Date Fri, 18 Jan 2019 14:20:04 GMT
Hi,

We have some Timeout frequently in our node cassandra, can any one help me
to analyse what can be the cause.


INFO  [SlabPoolCleaner] 2018-12-27 05:07:01,836 ColumnFamilyStore.java:1197
- Flushing largest CFS(Keyspace='db_cass_agregats',
ColumnFamily='customers_metrics') to free up room. Used total: 0.45/0.00,
live: 0.33/0.00, flushing: 0.11/0.00, this: 0.11/0.11
INFO  [SlabPoolCleaner] 2018-12-27 05:07:01,836 ColumnFamilyStore.java:905
- Enqueuing flush of customers_metrics: 239025519 (11%) on-heap, 0 (0%)
off-heap
INFO  [MemtableFlushWriter:55844] 2018-12-27 05:07:01,853 Memtable.java:347
- Writing Memtable-customers_metrics@2086879045(189.254MiB serialized
bytes, 163048 ops, 11%/0% of on/off-heap limit)
INFO  [MemtableFlushWriter:55845] 2018-12-27 05:07:02,398 Memtable.java:382
- Completed flushing
/var/opt/data/flat/bdfcas/files/data/db_cass_agregats/customers_metrics-af4be81092db11e78b314125c9bc7cec/db_cass_agregats-customers_metrics-tmp-ka-140634-Data.db
(21.863MiB) for commitlog position ReplayPosition(segmentId=1525681566111,
position=14599365)
INFO  [MemtableFlushWriter:55844] 2018-12-27 05:07:02,912 Memtable.java:382
- Completed flushing
/var/opt/data/flat/bdfcas/files/data/db_cass_agregats/customers_metrics-af4be81092db11e78b314125c9bc7cec/db_cass_agregats-customers_metrics-tmp-ka-140635-Data.db
(21.556MiB) for commitlog position ReplayPosition(segmentId=1525681566117,
position=19478541)
INFO  [SlabPoolCleaner] 2018-12-27 05:07:03,034 ColumnFamilyStore.java:1197
- Flushing largest CFS(Keyspace='db_cass_agregats',
ColumnFamily='customers_metrics') to free up room. Used total: 0.45/0.00,
live: 0.33/0.00, flushing: 0.11/0.00, this: 0.11/0.11
INFO  [SlabPoolCleaner] 2018-12-27 05:07:03,035 ColumnFamilyStore.java:905
- Enqueuing flush of customers_metrics: 239127661 (11%) on-heap, 0 (0%)
off-heap
INFO  [MemtableFlushWriter:55845] 2018-12-27 05:07:03,047 Memtable.java:347
- Writing Memtable-customers_metrics@962624348(189.403MiB serialized bytes,
163122 ops, 11%/0% of on/off-heap limit)
INFO  [SlabPoolCleaner] 2018-12-27 05:07:04,113 ColumnFamilyStore.java:1197
- Flushing largest CFS(Keyspace='db_cass_agregats',
ColumnFamily='customers_metrics') to free up room. Used total: 0.45/0.00,
live: 0.33/0.00, flushing: 0.11/0.00, this: 0.11/0.11
INFO  [SlabPoolCleaner] 2018-12-27 05:07:04,113 ColumnFamilyStore.java:905
- Enqueuing flush of customers_metrics: 239049872 (11%) on-heap, 0 (0%)
off-heap
INFO  [MemtableFlushWriter:55844] 2018-12-27 05:07:04,123 Memtable.java:347
- Writing Memtable-customers_metrics@219594445(189.231MiB serialized bytes,
163174 ops, 11%/0% of on/off-heap limit)
INFO  [MemtableFlushWriter:55845] 2018-12-27 05:07:04,210 Memtable.java:382
- Completed flushing
/var/opt/data/flat/bdfcas/files/data/db_cass_agregats/customers_metrics-af4be81092db11e78b314125c9bc7cec/db_cass_agregats-customers_metrics-tmp-ka-140636-Data.db
(21.873MiB) for commitlog position ReplayPosition(segmentId=1525681566123,
position=24516704)
INFO  [MemtableFlushWriter:55844] 2018-12-27 05:07:05,168 Memtable.java:382
- Completed flushing
/var/opt/data/flat/bdfcas/files/data/db_cass_agregats/customers_metrics-af4be81092db11e78b314125c9bc7cec/db_cass_agregats-customers_metrics-tmp-ka-140637-Data.db
(21.613MiB) for commitlog position ReplayPosition(segmentId=1525681566129,
position=29370537)
ERROR [SharedPool-Worker-1] 2018-12-27 05:07:10,349 QueryMessage.java:135 -
Unexpected error during query
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.getPermissions(Auth.java:75)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:251)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:245)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:229)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:132)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:235)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
[apache-cassandra-2.1.13.jar:2.1.13]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_75]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-2.1.13.jar:2.1.13]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:124)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:121)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
~[guava-16.0.jar:na]
... 23 common frames omitted
Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation
timed out - received only 2 responses.
at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:105)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:146)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1397)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1324)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1247)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:271)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:224)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:265)
~[apache-cassandra-2.1.13.jar:2.1.13]
... 32 common frames omitted
ERROR [SharedPool-Worker-2] 2018-12-27 05:07:10,349 QueryMessage.java:135 -
Unexpected error during query
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.getPermissions(Auth.java:75)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:251)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:245)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:229)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:132)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:235)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
[apache-cassandra-2.1.13.jar:2.1.13]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_75]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-2.1.13.jar:2.1.13]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:124)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:121)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
~[guava-16.0.jar:na]
... 23 common frames omitted
Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation
timed out - received only 2 responses.
at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:105)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:146)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1397)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1324)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1247)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:271)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:224)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:265)
~[apache-cassandra-2.1.13.jar:2.1.13]
... 32 common frames omitted
ERROR [SharedPool-Worker-4] 2018-12-27 05:07:10,349 QueryMessage.java:135 -
Unexpected error during query
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.getPermissions(Auth.java:75)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:251)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:245)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:229)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:132)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:235)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
[apache-cassandra-2.1.13.jar:2.1.13]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_75]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-2.1.13.jar:2.1.13]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:124)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:121)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
~[guava-16.0.jar:na]
... 23 common frames omitted
Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation
timed out - received only 2 responses.
at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:105)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:146)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1397)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1324)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1247)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:271)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:224)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:265)
~[apache-cassandra-2.1.13.jar:2.1.13]
... 32 common frames omitted
ERROR [SharedPool-Worker-5] 2018-12-27 05:07:10,350 QueryMessage.java:135 -
Unexpected error during query
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.getPermissions(Auth.java:75)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:251)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:245)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:229)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:132)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:235)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
[apache-cassandra-2.1.13.jar:2.1.13]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_75]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-2.1.13.jar:2.1.13]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:124)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:121)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
~[guava-16.0.jar:na]
... 23 common frames omitted
Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation
timed out - received only 2 responses.
at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:105)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:146)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1397)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1324)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1247)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:271)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:224)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:265)
~[apache-cassandra-2.1.13.jar:2.1.13]
... 32 common frames omitted
ERROR [SharedPool-Worker-1] 2018-12-27 05:07:10,350 ErrorMessage.java:251 -
Unexpected exception during request
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.getPermissions(Auth.java:75)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:251)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:245)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:229)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:132)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:235)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
[apache-cassandra-2.1.13.jar:2.1.13]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_75]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-2.1.13.jar:2.1.13]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:124)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:121)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
~[guava-16.0.jar:na]
... 23 common frames omitted
Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation
timed out - received only 2 responses.
at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:105)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:146)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1397)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1324)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1247)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:271)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:224)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:265)
~[apache-cassandra-2.1.13.jar:2.1.13]
... 32 common frames omitted
ERROR [SharedPool-Worker-2] 2018-12-27 05:07:10,350 ErrorMessage.java:251 -
Unexpected exception during request
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.getPermissions(Auth.java:75)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:251)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:245)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:229)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:132)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:235)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
[apache-cassandra-2.1.13.jar:2.1.13]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_75]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-2.1.13.jar:2.1.13]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:124)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:121)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
~[guava-16.0.jar:na]
... 23 common frames omitted
Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation
timed out - received only 2 responses.
at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:105)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:146)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1397)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1324)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1247)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:271)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:224)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:265)
~[apache-cassandra-2.1.13.jar:2.1.13]
... 32 common frames omitted
ERROR [SharedPool-Worker-4] 2018-12-27 05:07:10,351 ErrorMessage.java:251 -
Unexpected exception during request
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.getPermissions(Auth.java:75)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:251)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:245)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:229)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:132)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:235)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
[apache-cassandra-2.1.13.jar:2.1.13]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_75]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-2.1.13.jar:2.1.13]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:124)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:121)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
~[guava-16.0.jar:na]
... 23 common frames omitted
Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation
timed out - received only 2 responses.
at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:105)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:146)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1397)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1324)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1247)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:271)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:224)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:265)
~[apache-cassandra-2.1.13.jar:2.1.13]
... 32 common frames omitted
ERROR [SharedPool-Worker-5] 2018-12-27 05:07:10,351 ErrorMessage.java:251 -
Unexpected exception during request
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.getPermissions(Auth.java:75)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.authorize(ClientState.java:353)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:251)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:245)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:229)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.ModificationStatement.checkAccess(ModificationStatement.java:132)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:235)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:122)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
[apache-cassandra-2.1.13.jar:2.1.13]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_75]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-2.1.13.jar:2.1.13]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 2 responses.
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:276)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.isSuperuser(Auth.java:97)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:50)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.CassandraAuthorizer.authorize(CassandraAuthorizer.java:67)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:124)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.auth.PermissionsCache$1.load(PermissionsCache.java:121)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
~[guava-16.0.jar:na]
... 23 common frames omitted
Caused by: org.apache.cassandra.exceptions.ReadTimeoutException: Operation
timed out - received only 2 responses.
at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:105)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:146)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1397)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1324)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1247)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:271)
~[apache-cassandra-2.1.13.jar:2.1.13]
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:224)
~[apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.auth.Auth.selectUser(Auth.java:265)
~[apache-cassandra-2.1.13.jar:2.1.13]
... 32 common frames omitted
INFO  [SlabPoolCleaner] 2018-12-27 05:07:13,204 ColumnFamilyStore.java:1197
- Flushing largest CFS(Keyspace='db_cass_agregats',
ColumnFamily='customers_metrics') to free up room. Used total: 0.33/0.00,
live: 0.33/0.00, flushing: 0.00/0.00, this: 0.11/0.11
INFO  [SlabPoolCleaner] 2018-12-27 05:07:13,204 ColumnFamilyStore.java:905
- Enqueuing flush of customers_metrics: 239164849 (11%) on-heap, 0 (0%)
off-heap
INFO  [MemtableFlushWriter:55845] 2018-12-27 05:07:13,216 Memtable.java:347
- Writing Memtable-customers_metrics@1858853610(189.322MiB serialized
bytes, 163202 ops, 11%/0% of on/off-heap limit)
INFO  [SlabPoolCleaner] 2018-12-27 05:07:13,801 ColumnFamilyStore.java:1197
- Flushing largest CFS(Keyspace='db_cass_agregats',
ColumnFamily='customers_metrics') to free up room. Used total: 0.45/0.00,
live: 0.33/0.00, flushing: 0.11/0.00, this: 0.11/0.11
INFO  [SlabPoolCleaner] 2018-12-27 05:07:13,801 ColumnFamilyStore.java:905
- Enqueuing flush of customers_metrics: 239050988 (11%) on-heap, 0 (0%)
off-heap


Best
Rabii

Mime
View raw message