Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0197117622 for ; Tue, 10 Mar 2015 16:16:39 +0000 (UTC) Received: (qmail 34801 invoked by uid 500); 10 Mar 2015 16:16:36 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 34758 invoked by uid 500); 10 Mar 2015 16:16:36 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 34748 invoked by uid 99); 10 Mar 2015 16:16:36 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Mar 2015 16:16:36 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW X-Spam-Check-By: apache.org Received-SPF: error (athena.apache.org: local policy) Received: from [209.85.213.46] (HELO mail-yh0-f46.google.com) (209.85.213.46) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Mar 2015 16:16:31 +0000 Received: by yhai57 with SMTP id i57so1360697yha.6 for ; Tue, 10 Mar 2015 09:13:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:reply-to:sender:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=nmYu4w2dP0LvSUMgLd8s9nXSzbD6fV5DtXS2ijxEgqg=; b=LYUgwufeyTG+9DGBIt2I0LXD2ahna+JK6bvNbwUpbVazHSHtz9+ZAw245xcG/+PKWr YVXcVrtKVF92Ac6T9TCfQgIpcS+dL0wPwsK3v5ciIV7eBx+dOkHkjahWfOrMuiBkrVbh KgFdc0C4dXX/akz5TgwyF6wG5ilRV0OisQM3W3Vu15P1JEdYUjxwjJnN8Z6Uz8kfR/Dm KbWtj53oE32Xr4Jv4w8djV94nNaTkpZqxuZcWkDoZ+WH9DfAUo3obACvi+ekGbuAXRgy yBYEEpQkpQuRVO/Z+nlvntrYd2K1/E2qHoUu6YcwQe4kAqcV5CKlQF8R8ES1JYWf0M68 JeIg== X-Gm-Message-State: ALoCoQl7/w1KCwSwD7nTvszsX6Oxsd0Uk2CM8IDcmK/1/K0EWt3UY9P+WEwAcFQsM6v+meIAx7Pc MIME-Version: 1.0 X-Received: by 10.236.209.35 with SMTP id r23mr32126972yho.26.1426004015094; Tue, 10 Mar 2015 09:13:35 -0700 (PDT) Reply-To: mail@frensjan.nl Sender: frensjan@frensjan.nl Received: by 10.170.83.195 with HTTP; Tue, 10 Mar 2015 09:13:35 -0700 (PDT) X-Originating-IP: [89.188.21.236] In-Reply-To: References: <1425492417697.a19cfa28@Nodemailer> <1425495806878.c9686778@Nodemailer> Date: Tue, 10 Mar 2015 17:13:35 +0100 X-Google-Sender-Auth: b0tF35wWszgHuKdlLKnPwGxeNMM Message-ID: Subject: Re: Inconsistent count(*) and distinct results from Cassandra From: "Rumph, Frens Jan" To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001a11c20266c4b45d0510f16e8d X-Virus-Checked: Checked by ClamAV on apache.org --001a11c20266c4b45d0510f16e8d Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Jens, Mikhail, Daemeon, Thanks for your replies. Sorry for my reply being late ... mails from the user-list were moved to the wrong inbox on my side. I'm in a development environment and thus using replication factor =3D 1 an= d consistency =3D ONE with three nodes. So the 'results from different nodes between queries' hypothesis seems unlikely to me. I would expect a timeout if some node wouldn't be able to answer. I tried tracing, but I couldn't really make any of it. For example I performed two select distinct ... from ... queries: Traces for both of them contained more than one line like 'Submitting range requests on ... ranges ...' and 'Submitted ... concurrent range requests covering ... ranges'. These lines occur with varying numbers, e.g. : Submitting range requests on 593 ranges with a concurrency of 75 (1.35 rows per range expected) Submitting range requests on 769 ranges with a concurrency of 75 (1.35 rows per range expected) Also when looking at the lines like 'Executing seq scan across ... sstables for ...' I saw that in one case which yielded way less partition keys that only the tokens from -9223372036854770000 to -594461978511041000 were included. In a case which yielded much more partition keys, the entire token range did seem to be queried. To reiterate my initial questions: is this behavior to be expected? Am I doing something wrong? Is there a workaround? Best regards, Frens Jan On 4 March 2015 at 22:59, daemeon reiydelle wrote: > What is the replication? Could you be serving stale data from a node that > was not properly replicated (hints timeout exceeded by a node being down?= ) > > > > On Wed, Mar 4, 2015 at 11:03 AM, Jens Rantil wrote: > >> Frens, >> >> What consistency are you querying with? Could be you are simply receivin= g >> result from different nodes each time. >> >> Jens >> >> =E2=80=93 >> Skickat fr=C3=A5n Mailbox >> >> >> On Wed, Mar 4, 2015 at 7:08 PM, Mikhail Strebkov >> wrote: >> >>> We have observed the same issue in our production Cassandra cluster (5 >>> nodes in one DC). We use Cassandra 2.1.3 (I joined the list too late to >>> realize we shouldn=E2=80=99t user 2.1.x yet) on Amazon machines (create= d from >>> community AMI). >>> >>> In addition to count variations with 5 to 10% we observe variations for >>> the query =E2=80=9Cselect * from table1 where time > '$fromDate' and ti= me < >>> '$toDate' allow filtering=E2=80=9D results. We iterated through the res= ults >>> multiple times using official Java driver. We used that query for a hug= e >>> data migration and were unpleasantly surprised that it is unreliable. I= n >>> our case =E2=80=9Cnodetool repair=E2=80=9D didn=E2=80=99t fix the issue= . >>> >>> So I echo Frens questions. >>> >>> Thanks, >>> Mikhail >>> >>> >>> >>> >>> On Wed, Mar 4, 2015 at 3:55 AM, Rumph, Frens Jan >>> wrote: >>> >>>> Hi, >>>> >>>> Is it to be expected that select count(*) from ... and select distinct >>>> partition-key-columns from ... to yield inconsistent results between >>>> executions even though the table at hand isn't written to? >>>> >>>> I have a table in a keyspace with replication_factor =3D 1 which is >>>> something like: >>>> >>>> CREATE TABLE tbl ( >>>> id frozen, >>>> bucket bigint, >>>> offset int, >>>> value double, >>>> PRIMARY KEY ((id, bucket), offset) >>>> ) >>>> >>>> The frozen udt is: >>>> >>>> CREATE TYPE id_type ( >>>> tags map >>>> ); >>>> >>>> When I do select count(*) from tbl several times the actual count >>>> varies with 5 to 10%. Also when performing select distinct id, bucket = from >>>> tbl the results aren't consistent over several query executions. The t= able >>>> is not being written to at the time I performed the queries. >>>> >>>> Is this to be expected? Or is this a bug? Is there a alternative metho= d >>>> / workaround? >>>> >>>> I'm using cqlsh 5.0.1 with Cassandra 2.1.2 on 64bit fedora 21 with >>>> Oracle Java 1.8.0_31. >>>> >>>> Thanks in advance, >>>> Frens Jan >>>> >>> >>> >> > --001a11c20266c4b45d0510f16e8d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Jens, Mikh= ail, Daemeon,

Thanks for your replies. Sorry for my reply being late ... mails from the = user-list were moved to the wrong inbox on my side.

I'm in a development environment= and thus using replication factor =3D 1 and consistency =3D ONE with three= nodes. So the 'results from different nodes between queries' hypot= hesis seems unlikely to me. I would expect a timeout if some node wouldn= 9;t be able to answer.
I tried tracing, but I couldn't really make any of it.=

For example I perfo= rmed two select distinct ... from ... queries: Traces for both of them cont= ained more than one line like 'Submitting range requests on ... ranges = ...' and 'Submitted ... concurrent range requests covering ... rang= es'. These lines occur with varying numbers, e.g. :

Submitting r= ange requests on 593 ranges with a concurrency of 75 (1.35 rows per range e= xpected)=C2=A0
Submitting range requests on 769 ranges with a concurrenc= y of 75 (1.35 rows per range expected)
Also when l= ooking at the lines like 'Executing seq scan across ... sstables for ..= .' I saw that in one case which yielded way less partition keys that on= ly the tokens from -9223372036854770000=C2=A0 to -594461978511041000 were i= ncluded. In a case which yielded much more partition keys, the entire token= range did seem to be queried.

To reiterate my initial questions: is this behavior to be= expected? Am I doing something wrong? Is there a workaround?

Best regards,
Frens Jan

On 4 March 2015 at 22:59, daemeon reiydelle <daemeonr@gm= ail.com> wrote:
What is the replication? Could you be serving sta= le data from a node that was not properly replicated (hints timeout exceede= d by a node being down?)



On Wed, Mar 4, 2015 at 11:03 AM, Jens Rantil= <jens.rantil@tink.se> wrote:
Frens,

What consistency are you querying with? Could be you are= simply receiving result from different nodes each time.

Jens

=E2=80=93
Skickat fr=C3=A5n Mailbox


On Wed, Mar 4, 2015 at 7:08 PM, Mikha= il Strebkov <strebkov@gmail.com> wrote:

We have observed the same issue in our production Cassandra clus= ter (5 nodes in one DC). We use Cassandra 2.1.3 (I joined the list too late= to realize we shouldn=E2=80=99t user 2.1.x yet) on Amazon machines (create= d from community AMI).

In addition to count variations with 5 to 10% we observe variations fo= r the query =E2=80=9Cselect * from table1 where time > '$fromDate= 9; and time < '$toDate' allow filtering=E2=80=9D results. We ite= rated through the results multiple times using official Java driver. We use= d that query for a huge data migration and were unpleasantly surprised that= it is unreliable. In our case =E2=80=9Cnodetool repair=E2=80=9D didn=E2=80= =99t fix the issue.

So I echo Frens questions.

Thanks,
Mikhail




On Wed, Mar 4, 2015 at 3:55 AM, Rumph, Frens Jan <<= a href=3D"mailto:mail@frensjan.nl" target=3D"_blank">mail@frensjan.nl&g= t; wrote:

Hi,

Is it to be expected that select count(*) from ... and select distinct= partition-key-columns from ... to yield inconsistent results between execu= tions even though the table at hand isn't written to?

I have a table in a keyspace with replication_factor =3D 1 which is so= mething like:

CREATE TABLE tbl (
=C2=A0 =C2=A0 id frozen<id_type>,
=C2=A0 =C2=A0 bucket bigint,
=C2=A0 =C2=A0 offset int,
=C2=A0 =C2=A0 value double,
=C2=A0 =C2=A0 PRIMARY KEY ((id, bucket), offset)
)

The frozen udt is:

CREATE TYPE id_type (
=C2=A0 =C2=A0 tags map<text, text>
);

When I do select count(*) from tbl several times the actual count vari= es with 5 to 10%. Also when performing select distinct id, bucket from tbl = the results aren't consistent over several query executions. The table = is not being written to at the time I performed the queries.

Is this to be expected? Or is this a bug? Is there a alternative metho= d / workaround?

I'm using cqlsh 5.0.1 with Cassandra 2.1.2 on 64bit fedora 21 with= Oracle Java 1.8.0_31.

Thanks in advance,
Frens Jan



<= /div>

--001a11c20266c4b45d0510f16e8d--