cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdul Patel <abd786...@gmail.com>
Subject Re: Tracing in cassandra
Date Fri, 12 Oct 2018 16:57:51 GMT
Let me try with limit 11 ..we have 18 node cluster ..no nodes down..

On Friday, October 12, 2018, Nitan Kainth <nitankainth@gmail.com> wrote:

> Try query with partition key selection in where clause. But time for limit
> 11 shouldn’t fail. Are all nodes up? Do you see any corruption in ay
> sstable?
>
> Sent from my iPhone
>
> On Oct 12, 2018, at 11:40 AM, Abdul Patel <abd786.ap@gmail.com> wrote:
>
> Sean,
>
> here it is :
> CREATE TABLE Keyspave.tblname (
>     user_id bigint,
>     session_id text,
>     application_guid text,
>     last_access_time timestamp,
>     login_time timestamp,
>     status int,
>     terminated_by text,
>     update_time timestamp,
>     PRIMARY KEY (user_id, session_id)
> ) WITH CLUSTERING ORDER BY (session_id ASC)
>
> also they see timeouts with limit 11 as well, so is it better to remove
> with limit option ? or whats best to query such schema?
>
> On Fri, Oct 12, 2018 at 11:05 AM Durity, Sean R <
> SEAN_R_DURITY@homedepot.com> wrote:
>
>> Cross-partition = multiple partitions
>>
>>
>>
>> Simple example:
>>
>> Create table customer (
>>
>> Customerid int,
>>
>> Name text,
>>
>> Lastvisit date,
>>
>> Phone text,
>>
>> Primary key (customerid) );
>>
>>
>>
>> Query
>>
>> Select customerid from customer limit 5000;
>>
>>
>>
>> The query is asking for 5000 different partitions to be selected across
>> the cluster. This is a very EXPENSIVE query for Cassandra, especially as
>> the number of nodes goes up. Typically, you want to query a single
>> partition. Read timeouts are usually caused by queries that are selecting
>> many partitions or a very large partition. That is why a schema for the
>> involved table could help.
>>
>>
>>
>>
>>
>> Sean Durity
>>
>>
>>
>> *From:* Abdul Patel <abd786.ap@gmail.com>
>> *Sent:* Friday, October 12, 2018 10:04 AM
>> *To:* user@cassandra.apache.org
>> *Subject:* [EXTERNAL] Re: Tracing in cassandra
>>
>>
>>
>> Cpuld you elaborate cross partition query?
>>
>> On Friday, October 12, 2018, Durity, Sean R <SEAN_R_DURITY@homedepot.com>
>> wrote:
>>
>> I suspect you are doing a cross-partition query, which will not scale
>> well (as you can see). What is the schema for the table involved?
>>
>>
>>
>>
>>
>> Sean Durity
>>
>>
>>
>> *From:* Abdul Patel <abd786.ap@gmail.com>
>> *Sent:* Thursday, October 11, 2018 5:54 PM
>> *To:* adam@instaclustr.com
>> *Cc:* user@cassandra.apache.org
>> *Subject:* [EXTERNAL] Re: Tracing in cassandra
>>
>>
>>
>> Query :
>>
>> SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000;
>>
>> -Error: ReadTimeout: Error from server: code=1200 [Coordinator node
>> timed out waiting for replica nodes' responses] message="Operation timed
>> out - received only 0 responses." info={'received_responses': 0,
>> 'required_responses': 1, 'consistency': 'ONE'}
>>
>>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT
>> 5000; | 10.54.145.32 |           4020 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                     Preparing statement
>> | 10.54.145.32 |           5065 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                 Executing
>> single-partition query on roles | 10.54.145.32 |           6171
>> |                                   ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                            Acquiring
>> sstable references | 10.54.145.32 |           6362
>> |                                   ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
>> 10.54.145.32 |           6641 |
>> ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                           Key cache hit
>> for sstable 346 | 10.54.145.32 |           6955
>> |                                   ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                Bloom filter allows
>> skipping sstable 347 | 10.54.145.32 |           7202
>> |                                   ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                               Merged
>> data from memtables and 2 sstables | 10.54.145.32 |           7386
>> |                                   ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                       Read 1 live and 0
>> tombstone cells | 10.54.145.32 |           7519
>> |                                   ReadStage-2
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                               Executing single-partition
>> query on roles | 10.54.145.32 |           7826 |
>>             ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                            Acquiring
>> sstable references | 10.54.145.32 |           7924
>> |                                   ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones |
>> 10.54.145.32 |           8060 |
>> ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                           Key cache hit
>> for sstable 346 | 10.54.145.32 |           8137
>> |                                   ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                    Bloom filter allows skipping sstable
>> 347 | 10.54.145.32 |           8187 |
>> ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                               Merged data from memtables
>> and 2 sstables | 10.54.145.32 |           8318
>> |                                   ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                       Read 1 live and 0
>> tombstone cells | 10.54.145.32 |           8941
>> |                                   ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd
>> |
>>
>>    Read-repair DC_LOCAL | 10.54.145.32 |           9468 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                         reading data from /10.54.145.31
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>> | 10.54.145.32 |           9754 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                               Executing single-partition
>> query on session | 10.54.145.32 |          10310
>> |                                   ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                       reading digest
>> from /10.54.101.56
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>> | 10.54.145.32 |          10343 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                            Acquiring
>> sstable references | 10.54.145.32 |          10356
>> |                                   ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                         Key cache hit
>> for sstable 68642 | 10.54.145.32 |          10445
>> |                                   ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                               reading digest from /10.54.101.57
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>> | 10.54.145.32 |          10480 |
>> Native-Transport-Requests-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                        Bloom filter allows skipping
>> sstable 68397 | 10.54.145.32 |          10496 |
>> ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd
>> |
>> Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
>> 10.54.145.32 |          10547 |
>> ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                   Sending READ message
>> to /10.54.145.31
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>> | 10.54.145.32 |          10575 | MessagingService-Outgoing-/10.
>> 54.145.31-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                         Key cache hit
>> for sstable 68655 | 10.54.145.32 |          10631
>> |                                   ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                   Sending READ message
>> to /10.54.101.56
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>> | 10.54.145.32 |          10695 | MessagingService-Outgoing-/10.
>> 54.101.56-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                           Sending READ
>> message to /10.54.101.57
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>> | 10.54.145.32 |          10823 | MessagingService-Outgoing-/10.
>> 54.101.57-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                           Merged data from memtables and
>> 3 sstables | 10.54.145.32 |          10863 |
>> ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                      Read 13 live and 0
>> tombstone cells | 10.54.145.32 |          10914
>> |                                   ReadStage-4
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>>                                                READ message received
>> from /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.145.31 |            883 |       MessagingService-Incoming-/10.
>> 54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                                                READ message received
>> from /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.57 |            803 |       MessagingService-Incoming-/10.
>> 54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>                                        READ message received from /
>> 10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.56 |            961 |       MessagingService-Incoming-/10.
>> 54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>>                                                       Executing
>> single-partition query on session | 10.54.145.31 |           3208
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                                             Executing single-partition
>> query on session | 10.54.101.57 |           3258
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>>                                                            Acquiring
>> sstable references | 10.54.145.31 |           3483
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                                                            Acquiring
>> sstable references | 10.54.101.57 |           3549
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>>                                 Key cache hit for sstable 74713 |
>> 10.54.145.31 |           3732 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                                                   Key cache hit for
>> sstable 85409 | 10.54.101.57 |           3978 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>                                             Executing single-partition
>> query on session | 10.54.101.56 |           3497
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>>                                                         Key cache hit
>> for sstable 74445 | 10.54.145.31 |           3893 |
>>                     ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                                              Bloom filter allows
>> skipping sstable 85201 | 10.54.101.57 |           4193
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>>                                              Bloom filter allows
>> skipping sstable 74383 | 10.54.145.31 |           3975
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                       Skipped 0/3 non-slice-intersecting sstables, included
>> 0 due to tombstones | 10.54.101.57 |           4354
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba
>> |                                                                         Skipped
>> 0/3 non-slice-intersecting sstables, included 0 due to tombstones |
>> 10.54.145.31 |           4142 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                                                         Key cache hit
>> for sstable 85418 | 10.54.101.57 |           4512
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                                               Merged data from memtables
>> and 3 sstables | 10.54.101.57 |           4989
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>                                                            Acquiring
>> sstable references | 10.54.101.56 |           3751
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                                              Read 13 live and 0
>> tombstone cells | 10.54.101.57 |           5336
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>                                                                  Key
>> cache hit for sstable 101147 | 10.54.101.56 |           4091
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                                                      Read 13 live and 0
>> tombstone cells | 10.54.101.57 |           5586
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                                                     Enqueuing response
>> to /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.57 |           5745 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90
>> |
>>                                       Sending REQUEST_RESPONSE message
>> to /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.57 |           6473 | MessagingService-Outgoing-/10.
>> 54.145.32-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd
>> |
>>            REQUEST_RESPONSE message received from /10.54.101.57
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>> | 10.54.145.32 |             57 |       MessagingService-Incoming-/10.
>> 54.101.57
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>>                                         Merged data from memtables and 3
>> sstables | 10.54.145.31 |           6600 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>                                                        Key cache hit for
>> sstable 100896 | 10.54.101.56 |           6496
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                  Processing response
>> from /10.54.101.57
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.57&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=zYDbByQA6i7TWxl_e6119WE_qT7auXhn8qscmGcG40A&e=>
>> | 10.54.145.32 |            587 |
>>          RequestResponseStage-1
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>>                                                      Read 13 live and 0
>> tombstone cells | 10.54.145.31 |           6892
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>>                                                     Enqueuing response
>> to /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.145.31 |           7038 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                            REQUEST_RESPONSE message
>> received from /10.54.145.31
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>> | 10.54.145.32 |             15 |       MessagingService-Incoming-/10.
>> 54.145.31
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba
>> |
>>                                   Sending REQUEST_RESPONSE message to /
>> 10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.145.31 |           7641 | MessagingService-Outgoing-/10.
>> 54.145.32-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>                                             Bloom filter allows skipping
>> sstable 100887 | 10.54.101.56 |           6652
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                                                  Processing response
>> from /10.54.145.31
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.31&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=VFRGkFAneO-HMP4eVg8FrXVgiDy0bsL9Q2MS83r85Qw&e=>
>> | 10.54.145.32 |             95 |
>> RequestResponseStage-1
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>> Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones |
>> 10.54.101.56 |           6828 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>                                                Key cache hit for sstable
>> 101149 | 10.54.101.56 |           7005 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>                                                         Merged data from
>> memtables and 4 sstables | 10.54.101.56 |           9897
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>                                                      Read 13 live and 0
>> tombstone cells | 10.54.101.56 |          11969
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>                                                      Read 13 live and 0
>> tombstone cells | 10.54.101.56 |          12067
>> |                                   ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>                                                     Enqueuing response
>> to /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.56 |          12361 |
>> ReadStage-3
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c
>> |
>>               Sending REQUEST_RESPONSE message to /10.54.145.32
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.145.32&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=C9nP5qbwIq2GMTXiAQ9d9sLjvow_mpfX1OBfrNKaGZE&e=>
>> | 10.54.101.56 |          14161 | MessagingService-Outgoing-/10.
>> 54.145.32-Small
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd
>> |
>>                              REQUEST_RESPONSE message received from /
>> 10.54.101.56
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>> | 10.54.145.32 |             51 |       MessagingService-Incoming-/10.
>> 54.101.56
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__10.54.101.56&d=DwMGaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=c_lov1NuHMZehIFK1Ip8FBAGIcTvp1uQLrI6WuMoUVg&s=KoAVTaVcog1srnn8npMVq8wpXmZq95ibi870ELyivGM&e=>
>>
>> e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd
>> |
>>
>>

Mime
View raw message