Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=content-type
	:mime-version:subject:from:in-reply-to:date
	:content-transfer-encoding:message-id:references:to; q=dns; s=
	thelastpickle.com; b=pm56WTr9lAmJ10WZc7E0eaqvdi+uUvqCkCQHyOyoqPR
	UYQZCKbDwp5FdZ9OzH/KoSDyIWvc8SKET2l7XZ2zpj+Lbd/6S/tz17YGn2gi1vZf
	PiqTYGNmEMmtNxXUfhMfwL2KoxUmA2H4eL+tt1chwlKbfCxvC9OWapXNb1KwiIqg
	=
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1084)
Subject: Re: need some help with counters
From: aaron morton <aaron@thelastpickle.com>
In-Reply-To: <8894F90A-9E13-450B-ABB4-DAA269733F52@Holsman.NET>
Date: Mon, 13 Jun 2011 07:10:07 +1200
Content-Transfer-Encoding: quoted-printable
Message-Id: <9E38D4F9-AFE6-4C22-B9D4-1B97E55C6BB8@thelastpickle.com>
References: <19BA1832-2346-4FAB-B8CD-F9C25177F2E5@Holsman.NET>
 <BANLkTinYxOpFYdGLQrc-=kxY3cf8py77_w@mail.gmail.com>
 <C66E6DCE-DFB4-4466-8B2A-9FFB90D383BE@Holsman.NET>
 <BANLkTi=h+TQxiB1FkKxm2aztU4eNvUThhg@mail.gmail.com>
 <5E753A9D-21CC-429F-8EE5-291A6C98F2BE@holsman.net>
 <C75792CA-7D99-400C-991F-53538E7BBE68@thelastpickle.com>
 <8894F90A-9E13-450B-ABB4-DAA269733F52@Holsman.NET>
To: user@cassandra.apache.org

> I am wondering how to index on the most recent hour as well. (ie show =
me top 5 URLs type query)..=20

AFAIK thats not a great application for counters. You would need range =
support in the secondary indexes so you could get the first X rows =
ordered by a column value.=20

To be honest, depending on scale, I'd consider a sorted set in redis for =
that.=20

Hope that helps.=20
 =20
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 11 Jun 2011, at 00:36, Ian Holsman wrote:

>=20
> On Jun 9, 2011, at 10:04 PM, aaron morton wrote:
>=20
>> I may be missing something but could you use a column for each of the =
last 48 hours all in the same row for a url ?
>>=20
>> e.g.=20
>> {
>> 	"/url.com/hourly" : {
>> 		"20110609T01:00:00" : 456,
>> 		"20110609T02:00:00" : 4567,
>> 	}
>> }
>=20
> yes.. that would work better... I was storing all the different times =
in the same row.
> {
> 	"/url.com" : {
> 	 "H-20110609T01:00:00" : 456,
> 	 "H-0110609T02:00:00" : 4567,
> 	 "D-0110609" : 5678,
> 	}
> }
>=20
> I am wondering how to index on the most recent hour as well. (ie show =
me top 5 URLs type query)..=20
>=20
>>=20
>> Increment the current hour only. Delete the older columns either when =
a read detects there are old values or as a maintenance job. Or as part =
of writing values for the first 5 minutes of any hour.=20
>=20
> yes.. I thought of that. The problem with doing it on read is there =
may be a case where a old URL never gets read.. so it will just sit =
there taking up space.. the maintenance job is the route I went down.
>=20
>>=20
>> The row will get spread out over a lot of sstables which may reduce =
read speed. If this is a problem consider a separate CF with more =
aggressive GC and compaction settings.=20
>=20
> Thanks!
>>=20
>> Cheers
>>=20
>>=20
>> -----------------
>> Aaron Morton
>> Freelance Cassandra Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>=20
>> On 10 Jun 2011, at 09:28, Ian Holsman wrote:
>>=20
>>> So would doing something like storing it in reverse (so I know what =
to delete) work? Or is storing a million columns in a supercolumn =
impossible.=20
>>>=20
>>> I could always use a logfile and run the archiver off that as a =
worst case I guess.=20
>>> Would doing so many deletes screw up the db/cause other problems?
>>>=20
>>> ---
>>> Ian Holsman - 703 879-3128
>>>=20
>>> I saw the angel in the marble and carved until I set him free -- =
Michelangelo
>>>=20
>>> On 09/06/2011, at 4:22 PM, Ryan King <ryan@twitter.com> wrote:
>>>=20
>>>> On Thu, Jun 9, 2011 at 1:06 PM, Ian Holsman <hadoop@holsman.net> =
wrote:
>>>>> Hi Ryan.
>>>>> you wouldn't have your version of cassandra up on github would =
you??
>>>>=20
>>>> No, and the patch isn't in our version yet either. We're still =
working on it.
>>>>=20
>>>> -ryan
>>=20
>=20