Return-Path: 
 <user-return-29177-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org>
X-Original-To: apmail-cassandra-user-archive@www.apache.org
Delivered-To: apmail-cassandra-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 13743D316
	for <apmail-cassandra-user-archive@www.apache.org>;
 Thu, 27 Sep 2012 18:51:39 +0000 (UTC)
Received: (qmail 8848 invoked by uid 500); 27 Sep 2012 18:51:36 -0000
Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org
Received: (qmail 8804 invoked by uid 500); 27 Sep 2012 18:51:36 -0000
Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@cassandra.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@cassandra.apache.org>
List-Post: <mailto:user@cassandra.apache.org>
List-Id: <user.cassandra.apache.org>
Reply-To: user@cassandra.apache.org
Delivered-To: mailing list user@cassandra.apache.org
Received: (qmail 8795 invoked by uid 99); 27 Sep 2012 18:51:36 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Sep 2012 18:51:36 +0000
X-ASF-Spam-Status: No, hits=-0.7 required=5.0
	tests=FSL_RCVD_USER,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of synfinatic@gmail.com
 designates 209.85.219.44 as permitted sender)
Received: from [209.85.219.44] (HELO mail-oa0-f44.google.com) (209.85.219.44)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Sep 2012 18:51:31 +0000
Received: by oagn5 with SMTP id n5so2597383oag.31
        for <user@cassandra.apache.org>; Thu, 27 Sep 2012 11:51:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type;
        bh=OAeeqHWOHF5S+i7l8R30sSE3KguHB+xAPF0OPOsnuMw=;
        b=om+dEnVGBIZ9irxjUstmn1Axaa/0rjA7koygGgbQE53S3Cy3C/StKPpDxmEQQh5Ihx
         C1rWYmJNV10Ar2ggX9nLIIejMeQbG/Qx17LhLsE81aXpyACNI5APdZ6dcHG/GNODvMaT
         c/6jhFvldg8XP7gXbiuepTBvtLA1+7hxaCw8J/Rqx7y7XkC53BqZ/0zOYXXv4Gkbny4j
         boUqlluE26oZi8F7KCVQtncvthgJ3JauBNP9Yj0OfffzUVOMPG/H7TPBLvny6FVkfp0h
         zXIN7jbKrU+Hdw0MButuqA7fXvbGtKBZ7nGnQN/Meew1+ANUxJLMcPPWi+pqftOW62j8
         Ccjw==
Received: by 10.182.92.3 with SMTP id ci3mr3982615obb.0.1348771870419; Thu, 27
 Sep 2012 11:51:10 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.60.42.166 with HTTP; Thu, 27 Sep 2012 11:50:50 -0700 (PDT)
In-Reply-To: 
 <CABKQidtdp6JOp8KLWkzgOvj7JHEDKD9sF3o7kBE9ajtf7yaGVw@mail.gmail.com>
References: 
 <CABKQidt+BEK_T-CYD7bUXF3GTgoLtQDZ1sgTRnKJUppK3CKEBA@mail.gmail.com>
 <CC89B8E9.11F8D%Dean.Hiller@nrel.gov>
 <CANAZdzWkDCjuLXGaxAndiKs7t_4v6Tu1Sv=vOVcYuNdT8sDB6g@mail.gmail.com>
 <CABKQidtdp6JOp8KLWkzgOvj7JHEDKD9sF3o7kBE9ajtf7yaGVw@mail.gmail.com>
From: Aaron Turner <synfinatic@gmail.com>
Date: Thu, 27 Sep 2012 19:50:50 +0100
Message-ID: 
 <CANAZdzWJPb02d-nDZh9z5g8f0ha2ueGv918H+RzmeEu3Rrv_Nw@mail.gmail.com>
Subject: Re: 1000's of column families
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
X-Virus-Checked: Checked by ClamAV on apache.org

On Thu, Sep 27, 2012 at 7:35 PM, Marcelo Elias Del Valle
<mvallebr@gmail.com> wrote:
>
>
> 2012/9/27 Aaron Turner <synfinatic@gmail.com>
>>
>> How strict are your security requirements?  If it wasn't for that,
>> you'd be much better off storing data on a per-statistic basis then
>> per-device.  Hell, you could store everything in a single CF by using
>> a composite row key:
>>
>> <devicename>|<stat type>|<instance>
>>
>> But yeah, there isn't a hard limit for the number of CF's, but there
>> is overhead associated with each one and so I wouldn't consider your
>> design as scalable.  Generally speaking, hundreds are ok, but
>> thousands is pushing it.
>
>
> Aaron,
>
>     Imagine that instead of using a composite key in this case, you use a
> simple row key "instance_uuid". Then, to index data by devicename |
> start_type|instance you use another CF with this composite key or several
> CFs to index it.
>     Do you see any drawbacks in terms of performance?

Really that depends on the client side I think.  Ideally, you'd like
to have the client be able to be able to directly access the row by
name without looking it up in some index.  Basically if you have to
lookup up the instance_uuid that's another call to some datastore
which takes more time then generating the row key via it's composites.
 At least that's my opinion...

Of course there are times where using an instance_uuid makes a lot of
sense... like if you rename a device and want all your stats to move
to the new name.  Much easier to just update the mapping record then
reading & rewriting all your rows for that device.

In my project, we use a device_uuid (just a primary key stored in an
Oracle DB... long story!), but everything else is by name in our
composite row keys.


-- 
Aaron Turner
http://synfin.net/         Twitter: @synfinatic
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Windows
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
    -- Benjamin Franklin
"carpe diem quam minimum credula postero"