cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mircea Lemnaru (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-11314) Inconsistent select count(*)
Date Thu, 10 Mar 2016 07:53:40 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-11314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15188869#comment-15188869
] 

Mircea Lemnaru commented on CASSANDRA-11314:
--------------------------------------------

Hi [~blerer], I did some more tests over the last couple of days and found that there is not
really a cassandra issue , it's rather something like a usage issue. Let me describe what
I found ... 

So as I have mentioned before the cluster had 3 nodes node1 (seed node) ,node2,node3 and my
keyspace had replication factor = 2. Then I did more tests using the following scenario:

1. node1 UP , node2 DOWN, node3 DOWN , I tried to do the following query: select count(*)
from collected_data_day where collection_day=16462 --> this query did not execute properly
because node1 did not contain a replica of the row = 16462 this suggested to me that the data
must be ONLY on nodes node2 and node3

2. node1 UP, node2 UP, node3 DOWN - executing the same query ( select count(*) from collected_data_day
where collection_day=16462 ) 20 times consecutively resulted in exactly the same result ...
COUNT = 55300 ... strange right ? 

3. node1 UP, node2 DOWN, node3 UP - same as 2 and I got COUNT = 55677 - every time , 20 times.

So this rang some bells , it seems that row = 16642 has 2 replicas = but the replicas are
not identical , it seems that when the data was written the CONSISTENCY level was not properly
set , in the sense that it was left default , and that is ONE if I am not mistaking.

I then proceeded to drop the table and reimport the data. This time I imported the data with
CONSISTENCY = ALL and after this , all the counts were identical , no matter how many times
I ran the query. 

So yeah ,this must have been the issue. I am guessing this can be closed as not a real issue
/ bug :)

PS: One thing that confused me all along was the COUNT values I got while the replicas were
not identical ... if you remember , I got something like: 

Count (done on cassandra cluster): 55300
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55300
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55634
Count (done on cassandra cluster): 55530
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55342
Count (done on cassandra cluster): 55300
Count (done on cassandra cluster): 55630
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55297
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55352
Count (done on cassandra cluster): 55300
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55300

>From these results ... it's clear that 55300 came from counting the items from node1 and
that 55677 came from node3 .... but what about 55634, 55342, 55352 ? Where were these results
comming from ... ?

Anyway, I am glad this is not a bug but a configuration issue. Thank you.

> Inconsistent select count(*)
> ----------------------------
>
>                 Key: CASSANDRA-11314
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11314
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Local Write-Read Paths
>         Environment: Ununtu 14.04 LTS
>            Reporter: Mircea Lemnaru
>            Assignee: Benjamin Lerer
>         Attachments: testrun.log, vnodes_and_hosts
>
>
> Hello,
> I currently have this setup: 
> Cassandra 3.3 (Community edition downloaded from Datastax) installed on 3 nodes and I
have created this table:
> CREATE TABLE billing.collected_data_day (
>     collection_day int,
>     timestamp timestamp,
>     record_id uuid,
>     dimensions map<text, text>,
>     entity_id text,
>     measurements map<text, text>,
>     source_id text,
>     PRIMARY KEY (collection_day, timestamp, record_id)
> ) WITH CLUSTERING ORDER BY (timestamp ASC, record_id ASC)
>     AND bloom_filter_fp_chance = 0.01
>     AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
>     AND comment = ''
>     AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
>     AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
>     AND crc_check_chance = 1.0
>     AND dclocal_read_repair_chance = 0.1
>     AND default_time_to_live = 0
>     AND gc_grace_seconds = 864000
>     AND max_index_interval = 2048
>     AND memtable_flush_period_in_ms = 0
>     AND min_index_interval = 128
>     AND read_repair_chance = 0.0
>     AND speculative_retry = '99PERCENTILE';
> This table as you notice is partitioned by collection_day. This is because at the end
of the day we need to have fast access to all the data generated in a day. collection day
will be the x day from 1970
> In this table we have inserted roughly 12milion rows for testing purposes and we did
a simple count. As you can see the results vary ... 
> cqlsh:billing> select count(*) from collected_data_day where collection_day=16462;
>  count
> -------
>  55341
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where collection_day=16462;
>  count
> -------
>  55372
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where collection_day=16462;
>  count
> -------
>  55300
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where collection_day=16462;
>  count
> -------
>  55300
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where collection_day=16462;
>  count
> -------
>  55300
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where collection_day=16462;
>  count
> -------
>  55303
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where collection_day=16462;
>  count
> -------
>  55374
> (1 rows)
> I am running the query from the seed node of the cassandra cluster. As you can see most
of the results are varying and I don't know the reason for this. We are not writing anything
into the cluster at this time , we are only querying the cluster and only using this CQLSH.
> This is very similar to CASSANDRA-8940 but that is targeted for 2.1x
> Could it be that we are having the same issue in 3.3 ? 
> Please let me know what extra info I can provide



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message