Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of stuhood@mailtrust.com
 designates 207.97.245.141 as permitted sender)
Date: Fri, 16 Apr 2010 16:22:17 -0500 (CDT)
Subject: Re: Regarding Cassandra Scalability
From: "Stu Hood" <stu.hood@rackspace.com>
To: user@cassandra.apache.org
MIME-Version: 1.0
Content-Type: text/plain;charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Importance: Normal
In-Reply-To: <4BC8CCB9.5070204@gmail.com>
References: <i2y7e99ee5f1004152328x8fb29b12jc18e8e4dd38e995@mail.gmail.com>   
 <p2r7ca6fc441004160138lbed0a77ch35e28f0288b30368@mail.gmail.com>   
 <4BC88D97.7050104@gmail.com> <-6358312770452844291@unknownmsgid>  
 <y2q828083e71004161102ze1f9d8eav9fbf9b61426cfe22@mail.gmail.com>
 <-8331721885605240603@unknownmsgid> <4BC8CCB9.5070204@gmail.com>
Message-ID: <1271452937.428424011@192.168.2.229>

http://twitter.com/jromeh/status/12295736793=0A=0A-----Original Message----=
-=0AFrom: "Mike Gallamore" <mike.e.gallamore@googlemail.com>=0ASent: Friday=
, April 16, 2010 3:46pm=0ATo: user@cassandra.apache.org=0ASubject: Re: Rega=
rding Cassandra Scalability=0A=0AAlso people with 1M followers tend to have=
 "public" tweets, which means =0Areally I think it would be the same as sub=
scribing to a RSS feed or =0Awhatever. You aren't getting a local copy beca=
use you will "always" have =0Aaccess to the tweet as will everyone else. Al=
so tweets don't change =0AAFAIK so no point in having redundant copies.=0AO=
n 04/16/2010 01:42 PM, Peter Chang wrote:=0A> Yeah. I wasn't sure if Cassan=
dra was optimized for binary data=0A> especially since any site of that siz=
e will use a CDN. Interesting=0A> read though.=0A>=0A> I think 1K per tweet=
 is off by an order of magnitude considering they=0A> only allow 140 charac=
ters. Regardless the number of users with>  1MM=0A> is probably a handful. =
Also im guessing they purge data after a=0A> certain window (like 30 days f=
or example).=0A>=0A> Sent from my iPhone=0A>=0A>=0A> On Apr 16, 2010, at 12=
:02 PM, gabriele renzi<rff.rff@gmail.com>  wrote:=0A>=0A>    =0A>> On Fri, =
Apr 16, 2010 at 6:41 PM, Peter Chang<peter78@gmail.com>=0A>> wrote:=0A>>   =
   =0A>>> FB also does pics and movies so 1MB is way off depending on where=
=0A>>> they=0A>>> manage such binary data.=0A>>>        =0A>> apparently no=
t in cassandra http://www.facebook.com/note.php?note_id=3D76191543919=0A>>=
=0A>>      =0A>>> I do agree that 1MB of text alone is a lot of text=0A>>> =
which is more relevant in the case of Twitter. The only large thing=0A>>> y=
ou=0A>>> leave out is denormalization. Every tweet you write is likely=0A>>=
> denormalized=0A>>> across your followers to allow for quick read access.=
=0A>>>        =0A>> .. but considering many users have _millions_ of follow=
ers, this may=0A>> be quite a bit more data. Assuming 1k per tweet, this wo=
uld mean one=0A>> from @aplusk (4.7M followers) would take more than 4 giga=
bytes of=0A>> data. Assuming ten tweets a day, in one month he'd produce on=
e TB.=0A>>=0A>> I'd say they only store references (increasing number lists=
 can also=0A>> be encoded very cleverly), or in some other way I'm not smar=
t enough=0A>> to think of.=0A>>      =0A=0A