cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdul Patel <abd786...@gmail.com>
Subject Re: Tracing in cassandra
Date Thu, 11 Oct 2018 21:54:17 GMT
Query :
SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
-Error: ReadTimeout: Error from server: code=1200 [Coordinator node timed
out waiting for replica nodes' responses] message="Operation timed out -
received only 0 responses." info={'received_responses': 0,
'required_responses': 1, 'consistency': 'ONE'}

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd
|
Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT
5000; | 10.54.145.32 |           4020 |
Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd
|
                                                    Preparing statement |
10.54.145.32 |           5065 |
Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd
|

       Executing
single-partition query on roles | 10.54.145.32 |           6171
|                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd
|
Acquiring sstable references | 10.54.145.32 |           6362
|                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd
|
Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
10.54.145.32 |           6641 |
                            ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd
|
Key cache hit for sstable 346 | 10.54.145.32 |           6955
|                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd
|
               Bloom filter allows skipping sstable 347 | 10.54.145.32
|           7202 |                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd
|
                                                              Merged data
from memtables and 2 sstables | 10.54.145.32 |           7386
|                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd
|
Read 1 live and 0 tombstone cells | 10.54.145.32 |           7519
|                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd
|
Executing single-partition query on roles | 10.54.145.32 |           7826
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd
|
Acquiring sstable references | 10.54.145.32 |           7924
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd
|
Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
10.54.145.32 |           8060 |
ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd
|
                                                          Key cache hit for
sstable 346 | 10.54.145.32 |           8137
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd
|

                        Bloom
filter allows skipping sstable 347 | 10.54.145.32 |           8187
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd
|
Merged data from memtables and 2 sstables | 10.54.145.32 |           8318
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd
|
Read 1 live and 0 tombstone cells | 10.54.145.32 |           8941
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd
|
   Read-repair DC_LOCAL | 10.54.145.32 |           9468 |
Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd
|
                                        reading data from /10.54.145.31 |
10.54.145.32 |           9754 |
Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd
|

     Executing
single-partition query on session | 10.54.145.32 |          10310
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd
|
reading digest from /10.54.101.56 | 10.54.145.32 |          10343
|                   Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd
|
Acquiring sstable references | 10.54.145.32 |          10356
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd
|
Key cache hit for sstable 68642 | 10.54.145.32 |          10445
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd
|
                              reading digest from /10.54.101.57 |
10.54.145.32 |          10480 |
Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd
|
                                                                     Bloom
filter allows skipping sstable 68397 | 10.54.145.32 |          10496
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd
|
Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
10.54.145.32 |          10547 |
ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd
|
Sending READ message to /10.54.145.31 | 10.54.145.32 |          10575 |
MessagingService-Outgoing-/10.54.145.31-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd
|
Key cache hit for sstable 68655 | 10.54.145.32 |          10631
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd
|
          Sending READ message to /10.54.101.56 | 10.54.145.32 |
10695 | MessagingService-Outgoing-/10.54.101.56-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd
|
                                                          Sending READ
message to /10.54.101.57 | 10.54.145.32 |          10823 |
MessagingService-Outgoing-/10.54.101.57-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd
|

                               Merged
data from memtables and 3 sstables | 10.54.145.32 |          10863
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd
|
Read 13 live and 0 tombstone cells | 10.54.145.32 |          10914
|                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba
|
READ message received from /10.54.145.32 | 10.54.145.31 |            883
|       MessagingService-Incoming-/10.54.145.32

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90
|
READ message received from /10.54.145.32 | 10.54.101.57 |            803
|       MessagingService-Incoming-/10.54.145.32

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c
|
                                       READ message received from /
10.54.145.32 | 10.54.101.56 |            961 |
MessagingService-Incoming-/10.54.145.32

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba
|

             Executing
single-partition query on session | 10.54.145.31 |           3208
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90
|
Executing single-partition query on session | 10.54.101.57 |           3258
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba
|
Acquiring sstable references | 10.54.145.31 |           3483
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90
|
Acquiring sstable references | 10.54.101.57 |           3549
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba
|
                                Key cache hit for sstable 74713 |
10.54.145.31 |           3732 |
ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90
|

         Key
cache hit for sstable 85409 | 10.54.101.57 |           3978
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c
|
Executing single-partition query on session | 10.54.101.56 |           3497
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba
|
Key cache hit for sstable 74445 | 10.54.145.31 |           3893
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90
|
Bloom filter allows skipping sstable 85201 | 10.54.101.57 |           4193
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba
|
     Bloom filter allows skipping sstable 74383 | 10.54.145.31 |
3975 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90
|
                      Skipped 0/3 non-slice-intersecting sstables, included
0 due to tombstones | 10.54.101.57 |           4354
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba
|
                                                                      Skipped
0/3 non-slice-intersecting sstables, included 0 due to tombstones |
10.54.145.31 |           4142 |
ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90
|
Key cache hit for sstable 85418 | 10.54.101.57 |           4512
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90
|
Merged data from memtables and 3 sstables | 10.54.101.57 |           4989
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c
|
   Acquiring sstable references | 10.54.101.56 |           3751
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90
|
                                             Read 13 live and 0 tombstone
cells | 10.54.101.57 |           5336 |
ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c
|

                        Key
cache hit for sstable 101147 | 10.54.101.56 |           4091
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90
|
Read 13 live and 0 tombstone cells | 10.54.101.57 |           5586
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90
|
Enqueuing response to /10.54.145.32 | 10.54.101.57 |           5745
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90
|
Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.101.57
|           6473 | MessagingService-Outgoing-/10.54.145.32-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd
|
           REQUEST_RESPONSE message received from /10.54.101.57 |
10.54.145.32 |             57 |       MessagingService-Incoming-/
10.54.101.57

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba
|
                                                                      Merged
data from memtables and 3 sstables | 10.54.145.31 |           6600
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c
|
Key cache hit for sstable 100896 | 10.54.101.56 |           6496
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd
|
Processing response from /10.54.101.57 | 10.54.145.32 |            587
|                        RequestResponseStage-1

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba
|
Read 13 live and 0 tombstone cells | 10.54.145.31 |           6892
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba
|
            Enqueuing response to /10.54.145.32 | 10.54.145.31 |
7038 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd
|
                                           REQUEST_RESPONSE message
received from /10.54.145.31 | 10.54.145.32 |             15 |
MessagingService-Incoming-/10.54.145.31

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba
|

                       Sending
REQUEST_RESPONSE message to /10.54.145.32 | 10.54.145.31 |           7641 |
MessagingService-Outgoing-/10.54.145.32-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c
|
Bloom filter allows skipping sstable 100887 | 10.54.101.56 |           6652
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd
|
Processing response from /10.54.145.31 | 10.54.145.32 |             95
|                        RequestResponseStage-1

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c
|
Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones |
10.54.101.56 |           6828 |
ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c
|
                                               Key cache hit for sstable
101149 | 10.54.101.56 |           7005 |
ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c
|

               Merged
data from memtables and 4 sstables | 10.54.101.56 |           9897
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c
|
Read 13 live and 0 tombstone cells | 10.54.101.56 |          11969
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c
|
Read 13 live and 0 tombstone cells | 10.54.101.56 |          12067
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c
|
Enqueuing response to /10.54.145.32 | 10.54.101.56 |          12361
|                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c
|
              Sending REQUEST_RESPONSE message to /10.54.145.32 |
10.54.101.56 |          14161 |
MessagingService-Outgoing-/10.54.145.32-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd
|
                                                           REQUEST_RESPONSE
message received from /10.54.101.56 | 10.54.145.32 |             51 |
MessagingService-Incoming-/10.54.101.56

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd
|
Processing response from /10.54.101.56 | 10.54.145.32 |            206
|                        RequestResponseStage-7

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d32f1-cd9e-11e8-8e99-15807bff4dfd
| Parsing SELECT * FROM keysoace.tablenameWHERE status = 0 AND
token(user_id) >= token(769281382) AND token(user_id) <= token(769281382)
LIMIT 12; | 10.54.145.32 |            519 |
   Native-Transport-Requests-2

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d5a00-cd9e-11e8-8e99-15807bff4dfd
|
Preparing statement | 10.54.145.32 |            906 |
Native-Transport-Requests-2

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8110-cd9e-11e8-8e99-15807bff4dfd
|
     Executing single-partition query on roles | 10.54.145.32 |
1820 |                                   ReadStage-1

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8111-cd9e-11e8-8e99-15807bff4dfd
|
                                                                  Acquiring
sstable references | 10.54.145.32 |           1884
|                                   ReadStage-1

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8112-cd9e-11e8-8e99-15807bff4dfd
|
                                                                     Skipped
0/2 non-slice-intersecting sstables, included 0 due to tombstones |
10.54.145.32 |           1968 |
ReadStage-1

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8113-cd9e-11e8-8e99-15807bff4dfd
|
Key cache hit for sstable 346 | 10.54.145.32 |           2023
|                                   ReadStage-1

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8114-cd9e-11e8-8e99-15807bff4dfd
|
Bloom filter allows skipping sstable 347 | 10.54.145.32 |           2076
|                                   ReadStage-1

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8115-cd9e-11e8-8e99-15807bff4dfd
|
                      Merged data from memtables and 2 sstables |
10.54.145.32 |           2133 |
ReadStage-1

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8116-cd9e-11e8-8e99-15807bff4dfd
|

       Read
1 live and 0 tombstone cells | 10.54.145.32 |           2179
|                                   ReadStage-1

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8117-cd9e-11e8-8e99-15807bff4dfd
|
Executing single-partition query on roles | 10.54.145.32 |           2290
|                                   ReadStage-3

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8118-cd9e-11e8-8e99-15807bff4dfd
|
Acquiring sstable references | 10.54.145.32 |           2371
|                                   ReadStage-3

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65d8119-cd9e-11e8-8e99-15807bff4dfd
|
Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
10.54.145.32 |           2494 |
ReadStage-3

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65da820-cd9e-11e8-8e99-15807bff4dfd
|
                  Key cache hit for sstable 346 | 10.54.145.32 |
2560 |                                   ReadStage-3

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65da821-cd9e-11e8-8e99-15807bff4dfd
|
                                                       Bloom filter allows
skipping sstable 347 | 10.54.145.32 |           2618
|                                   ReadStage-3

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65da822-cd9e-11e8-8e99-15807bff4dfd
|

                               Merged
data from memtables and 2 sstables | 10.54.145.32 |           2689
|                                   ReadStage-3

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65da823-cd9e-11e8-8e99-15807bff4dfd
|
Read 1 live and 0 tombstone cells | 10.54.145.32 |           2750
|                                   ReadStage-3

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df640-cd9e-11e8-8e99-15807bff4dfd
|
Index mean cardinalities are status_idx:1. Scanning with status_idx. |
10.54.145.32 |           4656 |
Native-Transport-Requests-2

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df641-cd9e-11e8-8e99-15807bff4dfd
|
      Computing ranges to query | 10.54.145.32 |           4844
|                   Native-Transport-Requests-2

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df642-cd9e-11e8-8e99-15807bff4dfd
|                                              Submitting range requests on
1 ranges with a concurrency of 1 (2.9296876E-4 rows per range expected) |
10.54.145.32 |           5109 |
Native-Transport-Requests-2

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df643-cd9e-11e8-8e99-15807bff4dfd
|

                   Submitted
1 concurrent range requests | 10.54.145.32 |           5286
|                   Native-Transport-Requests-2

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df644-cd9e-11e8-8e99-15807bff4dfd
|
Executing read on keysoace.tablenameusing index status_idx | 10.54.145.32
|           5386 |                                   ReadStage-6

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df645-cd9e-11e8-8e99-15807bff4dfd
|
Executing single-partition query on session.status_idx | 10.54.145.32
|           5476 |                                   ReadStage-6

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df646-cd9e-11e8-8e99-15807bff4dfd
|
Acquiring sstable references | 10.54.145.32 |           5499
|                                   ReadStage-6

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65df647-cd9e-11e8-8e99-15807bff4dfd
|
                               Key cache hit for sstable 42279 |
10.54.145.32 |           5539 |
ReadStage-6

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65e1d50-cd9e-11e8-8e99-15807bff4dfd
|

        Key
cache hit for sstable 42267 | 10.54.145.32 |           5574
|                                   ReadStage-6

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65e1d51-cd9e-11e8-8e99-15807bff4dfd
|
Key cache hit for sstable 41890 | 10.54.145.32 |           5601
|                                   ReadStage-6

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65e1d52-cd9e-11e8-8e99-15807bff4dfd
|
Key cache hit for sstable 37159 | 10.54.145.32 |           5704
|                                   ReadStage-6



---MORE---

session_id                           | event_id
|
activity
             | source       | source_elapsed | thread

--------------------------------------+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+--------------+----------------+-----------------------------------------------

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | f65e1d53-cd9e-11e8-8e99-15807bff4dfd
|
  Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones
| 10.54.145.32 |           5800 |
ReadStage-6

f65d32f0-cd9e-11e8-8e99-15807bff4dfd | fc533b00-cd9e-11e8-8e99-15807bff4dfd
|

                  Timed
out; received 0 of 1 responses | 10.54.145.32 |       10000610
|                   Native-Transport-Requests-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76eac6e0-cd6c-11e8-af38-373d9c5cfc0c
| Parsing SELECT * FROM keysoace.tablenameWHERE status = 0 AND
token(user_id) >= token(371384583) AND token(user_id) <= token(371384583)
LIMIT 11\n; | 10.153.42.83 |           3819 |
Native-Transport-Requests-3

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76eaedf0-cd6c-11e8-af38-373d9c5cfc0c
|
Preparing statement | 10.153.42.83 |           5486 |
Native-Transport-Requests-3

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ebff60-cd6c-11e8-af38-373d9c5cfc0c
|
reading data from /10.153.42.84 | 10.153.42.83 |          12093
|                   Native-Transport-Requests-3

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ec4d80-cd6c-11e8-af38-373d9c5cfc0c
|
                                Sending READ message to /10.153.42.84 |
10.153.42.83 |          13801 |
MessagingService-Outgoing-/10.153.42.84-Small

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76edad10-cd6c-11e8-a6c8-79961efb2cb8
|

        READ
message received from /10.153.42.83 | 10.153.42.84 |            928 |
MessagingService-Incoming-/10.153.42.83

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ee7060-cd6c-11e8-a6c8-79961efb2cb8
|
Executing single-partition query on roles | 10.153.42.84 |           5919
|                                   ReadStage-7

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ee7061-cd6c-11e8-a6c8-79961efb2cb8
|
Acquiring sstable references | 10.153.42.84 |           6327 |
                                ReadStage-7

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ee9770-cd6c-11e8-a6c8-79961efb2cb8
|
Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
10.153.42.84 |           6802 |
ReadStage-7

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76eee590-cd6c-11e8-a6c8-79961efb2cb8
|
                                  Key cache hit for sstable 477 |
10.153.42.84 |           8557 |
ReadStage-7

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76eee591-cd6c-11e8-a6c8-79961efb2cb8
|
                                                                         Bloom
filter allows skipping sstable 476 | 10.153.42.84 |           8836
|                                   ReadStage-7

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76eee592-cd6c-11e8-a6c8-79961efb2cb8
|
Merged data from memtables and 2 sstables | 10.153.42.84 |           9252
|                                   ReadStage-7

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef0ca0-cd6c-11e8-a6c8-79961efb2cb8
|
Read 1 live and 0 tombstone cells | 10.153.42.84 |           9428 |
                          ReadStage-7

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef0ca1-cd6c-11e8-a6c8-79961efb2cb8
|
Enqueuing response to /10.153.42.83 | 10.153.42.84 |           9554
|                                   ReadStage-7

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef0ca2-cd6c-11e8-a6c8-79961efb2cb8
|
        Sending REQUEST_RESPONSE message to /10.153.42.83 | 10.153.42.84
|          10130 | MessagingService-Outgoing-/10.153.42.83-Small

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef33b0-cd6c-11e8-af38-373d9c5cfc0c
|
                                                       REQUEST_RESPONSE
message received from /10.153.42.84 | 10.153.42.83 |          32954 |
MessagingService-Incoming-/10.153.42.84

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef5ac0-cd6c-11e8-af38-373d9c5cfc0c
|
Processing response from /10.153.42.84 | 10.153.42.83 |          34053
|                        RequestResponseStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef5ac1-cd6c-11e8-af38-373d9c5cfc0c
|
reading data from /10.153.42.84 | 10.153.42.83 |          34499
|                   Native-Transport-Requests-3

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef5ac2-cd6c-11e8-af38-373d9c5cfc0c
|
Sending READ message to /10.153.42.84 | 10.153.42.83 |          34775 |
MessagingService-Outgoing-/10.153.42.84-Small

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef81d0-cd6c-11e8-a6c8-79961efb2cb8
|
           READ message received from /10.153.42.83 | 10.153.42.84
|             14 |       MessagingService-Incoming-/10.153.42.83

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef81d1-cd6c-11e8-a6c8-79961efb2cb8
|
                                                            Executing
single-partition query on roles | 10.153.42.84 |            399
|                                   ReadStage-4

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef81d2-cd6c-11e8-a6c8-79961efb2cb8
|
Acquiring sstable references | 10.153.42.84 |            521
|                                   ReadStage-4

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef81d3-cd6c-11e8-a6c8-79961efb2cb8
|
Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
10.153.42.84 |            607 |
              ReadStage-4

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76ef81d4-cd6c-11e8-a6c8-79961efb2cb8
|
Key cache hit for sstable 477 | 10.153.42.84 |            736
|                                   ReadStage-4

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e0-cd6c-11e8-a6c8-79961efb2cb8
|
     Bloom filter allows skipping sstable 476 | 10.153.42.84 |
810 |                                   ReadStage-4

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e0-cd6c-11e8-af38-373d9c5cfc0c
|
                                           REQUEST_RESPONSE message
received from /10.153.42.84 | 10.153.42.83 |          36504 |
MessagingService-Incoming-/10.153.42.84

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e1-cd6c-11e8-a6c8-79961efb2cb8
|

                                 Merged
data from memtables and 2 sstables | 10.153.42.84 |           1021
|                                   ReadStage-4

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e1-cd6c-11e8-af38-373d9c5cfc0c
|
Processing response from /10.153.42.84 | 10.153.42.83 |          36729
|                        RequestResponseStage-5

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e2-cd6c-11e8-a6c8-79961efb2cb8
|
Read 1 live and 0 tombstone cells | 10.153.42.84 |           1132
|                                   ReadStage-4

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e3-cd6c-11e8-a6c8-79961efb2cb8
|
    Enqueuing response to /10.153.42.83 | 10.153.42.84 |           1180
|                                   ReadStage-4

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76efa8e4-cd6c-11e8-a6c8-79961efb2cb8
|
                                        Sending REQUEST_RESPONSE message to
/10.153.42.83 | 10.153.42.84 |           1279 |
MessagingService-Outgoing-/10.153.42.83-Small

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f0ba50-cd6c-11e8-af38-373d9c5cfc0c
|
                                                                       Index
mean cardinalities are status_idx:1. Scanning with status_idx. |
10.153.42.83 |          43356 |
Native-Transport-Requests-3

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f0ba51-cd6c-11e8-af38-373d9c5cfc0c
|
Computing ranges to query | 10.153.42.83 |          43791
|                   Native-Transport-Requests-3

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f0e160-cd6c-11e8-af38-373d9c5cfc0c
|                                                Submitting range requests
on 1 ranges with a concurrency of 1 (2.9296876E-4 rows per range expected)
| 10.153.42.83 |          44274 |
Native-Transport-Requests-3

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f0e161-cd6c-11e8-af38-373d9c5cfc0c
|
Enqueuing request to /10.153.42.85 | 10.153.42.83 |          44631
|                   Native-Transport-Requests-3

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f10870-cd6c-11e8-af38-373d9c5cfc0c
|
                                              Submitted 1 concurrent range
requests | 10.153.42.83 |          45063 |
Native-Transport-Requests-3

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f10871-cd6c-11e8-af38-373d9c5cfc0c
|

                  Sending
RANGE_SLICE message to /10.153.42.85 | 10.153.42.83 |          45074 |
MessagingService-Outgoing-/10.153.42.85-Small

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f1cbc0-cd6c-11e8-9866-ade6e01c2040
|
RANGE_SLICE message received from /10.153.42.83 | 10.153.42.85 |
605 |       MessagingService-Incoming-/10.153.42.83

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f1f2d0-cd6c-11e8-9866-ade6e01c2040
|
Executing read on keysoace.tablenameusing index status_idx | 10.153.42.85
|           2375 |                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f1f2d1-cd6c-11e8-9866-ade6e01c2040
|
Executing single-partition query on session.status_idx | 10.153.42.85
|           2599 |                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f1f2d2-cd6c-11e8-9866-ade6e01c2040
|
                                                Acquiring sstable
references | 10.153.42.85 |           2709
|                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f219e0-cd6c-11e8-9866-ade6e01c2040
|

Partition
index with 456 entries found for sstable 1386 | 10.153.42.85 |
2954 |                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f219e1-cd6c-11e8-9866-ade6e01c2040
|
Partition index with 11352 entries found for sstable 1367 | 10.153.42.85
|           3240 |                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f219e2-cd6c-11e8-9866-ade6e01c2040
|
Partition index with 16589 entries found for sstable 838 | 10.153.42.85
|           3413 |                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 76f219e3-cd6c-11e8-9866-ade6e01c2040
|
Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
10.153.42.85 |           3599 |
ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 7cdf4850-cd6c-11e8-af38-373d9c5cfc0c
|
                                  Timed out; received 0 of 1 responses |
10.153.42.83 |        9995766 |
Native-Transport-Requests-3

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a5aa0-cd6c-11e8-9866-ade6e01c2040
|

      Executing
single-partition query on session | 10.153.42.85 |       45774985
|                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a5aa1-cd6c-11e8-9866-ade6e01c2040
|
Acquiring sstable references | 10.153.42.85 |       45775171
|                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a5aa2-cd6c-11e8-9866-ade6e01c2040
|
Merging memtable contents | 10.153.42.85 |       45775241 |
                                 ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a5aa3-cd6c-11e8-9866-ade6e01c2040
|
Merging data from sstable 2489 | 10.153.42.85 |       45775454
|                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a5aa4-cd6c-11e8-9866-ade6e01c2040
|
           Partition index with 0 entries found for sstable 2489 |
10.153.42.85 |       45775602 |
ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a81b0-cd6c-11e8-9866-ade6e01c2040
|

             Merging
data from sstable 2455 | 10.153.42.85 |       45775894
|                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923a81b1-cd6c-11e8-9866-ade6e01c2040
|
Partition index with 2 entries found for sstable 2455 | 10.153.42.85
|       45776026 |                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923aa8c0-cd6c-11e8-9866-ade6e01c2040
|
Merging data from sstable 2335 | 10.153.42.85 |       45776836 |
                           ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923aa8c1-cd6c-11e8-9866-ade6e01c2040
|
Key cache hit for sstable 2335 | 10.153.42.85 |       45776958
|                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923be140-cd6c-11e8-9866-ade6e01c2040
|
                            Merging data from sstable 2293 | 10.153.42.85
|       45785346 |                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923be141-cd6c-11e8-9866-ade6e01c2040
|

       Key
cache hit for sstable 2293 | 10.153.42.85 |       45785532
|                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923ccba0-cd6c-11e8-9866-ade6e01c2040
|
Read 0 live and 0 tombstone cells | 10.153.42.85 |       45791600
|                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923cf2b0-cd6c-11e8-9866-ade6e01c2040
|
Merged data from memtables and 3 sstables | 10.153.42.85 |       45791761
|                                   ReadStage-2

76e98e60-cd6c-11e8-af38-373d9c5cfc0c | 923cf2b1-cd6c-11e8-9866-ade6e01c2040
|
Discarding partial response to /10.153.42.83 (timed out) | 10.153.42.85
|       45791858 |                                   ReadStage-2



On Thu, Oct 11, 2018 at 4:36 PM Adam Zegelin <adam@instaclustr.com> wrote:

> Hi Abdul,
>
> Can you post the timeout error and also the trace output?
>
> The trace output is detailed, but often refers to Cassandra internals. If
> we can see the error and the trace it'd be easier to pin-point the problem.
>
> - Adam
>
>
> On Thu, 11 Oct 2018 at 08:53, Abdul Patel <abd786.ap@gmail.com> wrote:
>
>> Hi,
>>
>> We have multiple timeouts with select queries.
>> I ran trace to see why its timing out ..do we jave a good document to
>> interpretate whats causing the queries to timeout?
>>
>

Mime
View raw message