Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of Jason.Alexander@match.com
 designates 208.185.193.221 as permitted sender)
From: Jason Alexander <Jason.Alexander@match.com>
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Date: Wed, 7 Apr 2010 21:28:43 -0500
Subject: RE: Integrity of batch_insert and also what about sharding?
Thread-Topic: Integrity of batch_insert and also what about sharding?
Thread-Index: AcrWv/dLg/UxDj6SQQiT+qkuBdtS8QAAosng
Message-ID: <EC6DD7F7E5C3A14D89A98EA82C490E503236A9E6A2@DA0ITMC101.match.corp>
References: <m2l3df6dbaf1004071541h81ac7646vcd9045de75157b5b@mail.gmail.com>
	 <u2ha7fcf8301004071607s8c46ad0ey111cb62cdde36ba0@mail.gmail.com>
	 <h2v3df6dbaf1004071802j75b50854o34d6162476904b7a@mail.gmail.com>
	 <t2ja7fcf8301004071840n7394a08ayf1254e4bde72a137@mail.gmail.com>
	 <i2q3df6dbaf1004071847yd5e76cc9m69a27d473bde01fd@mail.gmail.com>
	 <EC6DD7F7E5C3A14D89A98EA82C490E503236575E4C@DA0ITMC101.match.corp>
 <i2xa7fcf8301004071904n6d2651a0ja68df15b754a26f3@mail.gmail.com>
In-Reply-To: <i2xa7fcf8301004071904n6d2651a0ja68df15b754a26f3@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Well, IANAITG (I Am Not An IT Guy), but outside of the normal benefits you =
get from a SAN (that you can, of course, get from other options) is that I =
believe our IT group likes it for the management aspects - they like to buy=
 a BigAssSAN(tm) and provision storage to different clusters, environments,=
 etc... I'm sure it's also heavily weighted by the fact that it's "the devi=
l we know".=20

-----Original Message-----
From: Benjamin Black [mailto:b@b3k.us]=20
Sent: Wednesday, April 07, 2010 9:05 PM
To: user@cassandra.apache.org
Subject: Re: Integrity of batch_insert and also what about sharding?

What benefit does a SAN give you?  I've generally been confused by
that approach, so I'm assuming I am missing something.

On Wed, Apr 7, 2010 at 6:58 PM, Jason Alexander
<Jason.Alexander@match.com> wrote:
> FWIW, I'd love to see some guidance here too -
>
> From our standpoint, we'll be consolidating the various Match.com sites' =
(match.com, chemistry.com, etc...) data into a single data warehouse, runni=
ng Cassandra. We're looking at roughly the same amounts of data (30TB's or =
more). We were assuming 3-5 big servers sitting atop a SAN. But, again, jus=
t a guess and following existing conventions we use for other systems.
>
>
> ________________________________________
> From: banks [banksenus@gmail.com]
> Sent: Wednesday, April 07, 2010 8:47 PM
> To: user@cassandra.apache.org
> Subject: Re: Integrity of batch_insert and also what about sharding?
>
> What I'm trying to wrap my head around is what is the break even point...
>
> If I'm going to store 30terabytes in this thing... whats optimum to give =
me performance and scalability... is it best to be running 3 powerfull node=
s, 100 smaller nodes, nodes on each web blade with 300g behind each... =A0y=
a know? =A0I'm sure there is a point where the gossip chatter becomes overw=
elming and ups and downs to each... I have not really seen a best practices=
 document that gives the pro's and con's to each method of scaling.
>
> one 64proc 90gig memory mega machine running a single node cassandra... b=
ut on a raid5 SAN, good? bad? =A0why?
>
> 30 web blades each running a cassandra node, each with 1tb local raid5 st=
orage, good, bad, why?
>
> I get that every implimentation is different, what I'm looking for is wha=
t the known proven optimum is for this software... and whats to be avoided =
because its a given that it dosnt work.
>
> On Wed, Apr 7, 2010 at 6:40 PM, Benjamin Black <b@b3k.us<mailto:b@b3k.us>=
> wrote:
> That depends on your goals for fault tolerance and recovery time. =A0If
> you use RAID1 (or other redundant configuration) you can tolerate disk
> failure without Cassandra having to do repair. =A0For large data sets,
> that can be a significant win.
>
>
> b
>
> On Wed, Apr 7, 2010 at 6:02 PM, banks <banksenus@gmail.com<mailto:banksen=
us@gmail.com>> wrote:
>> Then from an IT standpoint, if i'm using a RF of 3, it stands to reason =
that
>> running on Raid 1 makes sense, since RAID and RF achieve the same ends..=
. it
>> makes sense to strip for speed and let cassandra deal with redundancy, e=
h?
>>
>>
>> On Wed, Apr 7, 2010 at 4:07 PM, Benjamin Black <b@b3k.us<mailto:b@b3k.us=
>> wrote:
>>>
>>> On Wed, Apr 7, 2010 at 3:41 PM, banks <banksenus@gmail.com<mailto:banks=
enus@gmail.com>> wrote:
>>> >
>>> > 2. each cassandra node essentially has the same datastore as all node=
s,
>>> > correct?
>>>
>>> No. =A0The ReplicationFactor you set determines how many copies of a
>>> piece of data you want. =A0If your number of nodes is higher than your
>>> RF, as is common, you will not have the same data on all nodes. =A0The
>>> exact set of nodes to which data is replicated is determined by the
>>> row key, placement strategy, and node tokens.
>>>
>>> > So if I've got 3 terabytes of data and 3 cassandra nodes I'm
>>> > eating 9tb on the SAN? =A0are there provisions for essentially shardi=
ng
>>> > across
>>> > nodes... so that each node only handles a given keyrange, if so where=
 is
>>> > the
>>> > howto on that?
>>> >
>>>
>>> Sharding is a concept from databases that don't have native
>>> replication and so need a term to describe what they bolt on for the
>>> functionality. =A0Distribution amongst nodes based on key ranges is how
>>> Cassandra always operates.
>>>
>>>
>>> b
>>
>>
>
>