Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of synfinatic@gmail.com
 designates 209.85.213.172 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <EE0F03BB-2E98-4D76-8DF3-84134F918907@thelastpickle.com>
References: 
 <CAAp6UWCUU78nyzB9tkaU-GvprxJ-P14gUxWN9_q0DbxOtW13iQ@mail.gmail.com>
 <CAKkz8Q0kh=zPpQZth9W5jhDZGW-ii7Sf9W8W8RrdeaEOS8PE3w@mail.gmail.com>
 <CAAnh3_-VV7Jc-0y-h5zLYrBViPy3pJ+L18qUZAudTYDMg=a6YA@mail.gmail.com>
 <4E29A50C.9080903@ihep.ac.cn>
 <CANAZdzV0JxSYK0JX5RfuyLWQ0nFjD2KHf-98sh0bYNY4ZJrBtw@mail.gmail.com>
 <EE0F03BB-2E98-4D76-8DF3-84134F918907@thelastpickle.com>
From: Aaron Turner <synfinatic@gmail.com>
Date: Mon, 25 Jul 2011 10:35:48 -0700
Message-ID: 
 <CANAZdzWPKN3dAGJeQK+JFrEscPx8hw0_K=JWwOO+UfKpMpYALQ@mail.gmail.com>
Subject: Re: Counter consistency - are counters idempotent?
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Sun, Jul 24, 2011 at 3:36 PM, aaron morton <aaron@thelastpickle.com> wro=
te:
> What's your use case ? There are people out there having good times with =
counters, see
>
> http://www.slideshare.net/kevinweil/rainbird-realtime-analytics-at-twitte=
r-strata-2011
> http://www.scribd.com/doc/59830692/Cassandra-at-Twitter

It's actually pretty similar to Twitter's click counting, but
apparently we have different requirements for accuracy.  It's possible
Rainbird does something on the front end to solve for this issue- I'm
honestly not sure since they haven't released the code yet.

Anyways, when you're building network aggregate graphs and fail to add
the +100G of traffic from one switch to your site or metro aggregate,
people around here notice.  And people quickly start distrusting
graphs which don't look "real" and either ignore them completely or
complain.

Obviously, one should manage their Cassandra cluster to limit the
occurrence of Timeouts, but frankly I don't want to be paged at 2am to
"fix" these kind of problems.  If I knew "timeout" meant "failed to
increment counter", I could spool my changes on the client and try
again later, but that's not what timeout means.  Without any means to
recover I've actually lost a lot of reliability that I currently have
with my single PostgreSQL server backed data store.

Right now I'm trying to come up with a way that my distributed snmp
pollers can build aggregates efficiently without counters, but that's
going to add a lot of complexity. :(

--=20
Aaron Turner
http://synfin.net/=A0 =A0 =A0 =A0=A0 Twitter: @synfinatic
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Win=
dows
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
=A0 =A0 -- Benjamin Franklin
"carpe diem quam minimum credula postero"