Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 51F469DA3 for ; Wed, 30 Nov 2011 21:57:04 +0000 (UTC) Received: (qmail 72438 invoked by uid 500); 30 Nov 2011 21:57:02 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 72412 invoked by uid 500); 30 Nov 2011 21:57:02 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 72404 invoked by uid 99); 30 Nov 2011 21:57:02 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Nov 2011 21:57:02 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.215.172] (HELO mail-ey0-f172.google.com) (209.85.215.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Nov 2011 21:56:55 +0000 Received: by eabm6 with SMTP id m6so1457047eab.31 for ; Wed, 30 Nov 2011 13:56:32 -0800 (PST) MIME-Version: 1.0 Received: by 10.216.229.74 with SMTP id g52mr80179weq.105.1322690192278; Wed, 30 Nov 2011 13:56:32 -0800 (PST) Received: by 10.180.8.232 with HTTP; Wed, 30 Nov 2011 13:56:32 -0800 (PST) In-Reply-To: <4ED6A40A.2010008@syncopated.net> References: <4ED698C0.60108@syncopated.net> <4ED6A40A.2010008@syncopated.net> Date: Wed, 30 Nov 2011 15:56:32 -0600 Message-ID: Subject: Re: data modeling question From: David McNelis To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=0016e64c056e0f060d04b2facf3c X-Virus-Checked: Checked by ClamAV on apache.org --0016e64c056e0f060d04b2facf3c Content-Type: text/plain; charset=ISO-8859-1 You wouldn't query for all the keys that have a column name x exactly. Instead what you would do is for sector x grab your list of symbols S. Then you would get the last column for each of those symbols (which you do in different ways depending on the API), and then test if that date is within your threshold. If not, it goes into your list of symbols to fetch. Alternatively, you could iterate over the symbols grabbing data where the date is between range A and B, if you get an empty set / no columns returned, then you need to re-pull for that symbol. Does that make sense? Either way you end up hitting on each of the individual symbols. Maybe someone else has a better idea of how to structure the data for that particular use case. On Wed, Nov 30, 2011 at 3:45 PM, Deno Vichas wrote: > with the quote CF below how would one query for all keys that have a > column name value that have a timeuuid of later than x minutes? i need to > be able to find all symbols that have not been fetch in x minutes by > sector. i know i get list of symbol by sector from my sector CF. > > thanks, > deno > > > On 11/30/2011 1:07 PM, David McNelis wrote: > >> >> Then I would have a column family for quotes where I have the key as the >> symbol, the column name as the timestamp, the value as the quote: >> >> quote : { >> key: symbol >> column names: timeuuid >> column values: quote at that time for that symbol >> } >> >> >> > -- *David McNelis* Lead Software Engineer Agentis Energy www.agentisenergy.com c: 219.384.5143 *A Smart Grid technology company focused on helping consumers of energy control an often under-managed resource.* --0016e64c056e0f060d04b2facf3c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable You wouldn't query for all the keys that have a column name x exactly. = =A0Instead what you would do is for sector x grab your list of symbols S. = =A0Then you would get the last column for each of those symbols (which you = do in different ways depending on the API), and then test if that date is w= ithin your threshold. =A0If not, it goes into your list of symbols to fetch= . =A0

Alternatively, you could iterate over the symbols grabbing d= ata where the date is between range A and B, if you get an empty set / no c= olumns returned, then you need to re-pull for that symbol. =A0Does that mak= e sense?

Either way you end up hitting on each of the individual symb= ols. =A0Maybe someone else has a better idea of how to structure the data f= or that particular use case.

On Wed, Nov 30, 2011 at 3:45 PM, Deno Vichas <deno@syncopated.net> wrote:
with the quote CF below how would one query for all keys that have a column= name value that have a timeuuid of later than x minutes? =A0i need to be a= ble to find all symbols that have not been fetch in x minutes by sector. = =A0i know i get list of symbol by sector from my sector CF.

thanks,
deno


On 11/30/2011 1:07 PM, David McNelis wrote:

Then I would have a column family for quotes where I have the key as the sy= mbol, the column name as the timestamp, the value as the quote:

quote : {
=A0 =A0key: symbol
=A0 =A0column names: =A0timeuuid
=A0 =A0column values: =A0quote at that time for that symbol
}






--
= David McNelis
Lead Software E= ngineer
Agentis Energy<= /font>
c: 219.384.5143

A Smar= t Grid technology company focused on helping consumers of energy control an= often under-managed resource.


--0016e64c056e0f060d04b2facf3c--