Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of yiming.sun@gmail.com
 designates 209.85.212.178 as permitted sender)
MIME-Version: 1.0
From: Yiming Sun <yiming.sun@gmail.com>
Date: Wed, 28 Mar 2012 16:40:47 -0400
Message-ID: 
 <CABxBLH95PKDdXAURD62_YJrJaBeNE81-P_5U0OUm5Ldpk5imaA@mail.gmail.com>
Subject: data size difference between supercolumn and regular column
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=f46d044304427991bf04bc53a083

--f46d044304427991bf04bc53a083
Content-Type: text/plain; charset=ISO-8859-1

Hi,

We are trying to estimate the amount of storage we need for a production
cassandra cluster.  While I was doing the calculation, I noticed a very
dramatic difference in terms of storage space used by cassandra data files.

Our previous setup consists of a single-node cassandra 0.8.x with no
replication, and the data is stored using supercolumns, and the data files
total about 534GB on disk.

A few weeks ago, I put together a cluster consisting of 3 nodes running
cassandra 1.0 with replication factor of 2, and the data is flattened out
and stored using regular columns.  And the aggregated data file size is
only 488GB (would be 244GB if no replication).

This is a very dramatic reduction in terms of storage needs, and is
certainly good news in terms of how much storage we need to provision.
 However, because of the dramatic reduction, I also would like to make sure
it is absolutely correct before submitting it - and also get a sense of why
there was such a difference. -- I know cassandra 1.0 does data compression,
but does the schema change from supercolumn to regular column also help
reduce storage usage?  Thanks.

-- Y.

--f46d044304427991bf04bc53a083
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi,<div><br></div><div>We are trying to estimate the amount of storage we n=
eed for a production cassandra cluster. =A0While I was doing the calculatio=
n, I noticed a very dramatic difference in terms of storage space used by c=
assandra data files.</div>

<div><br></div><div>Our previous setup consists of a single-node cassandra =
0.8.x with no replication, and the data is stored using supercolumns, and t=
he data files total about 534GB on disk.</div><div><br></div><div>A few wee=
ks ago, I put together a cluster consisting of 3 nodes running cassandra 1.=
0 with replication factor of 2, and the data is flattened out and stored us=
ing regular columns. =A0And the aggregated data file size is only 488GB (wo=
uld be 244GB if no replication).</div>

<div><br></div><div>This is a very dramatic reduction in terms of storage n=
eeds, and is certainly good news in terms of how much storage we need to pr=
ovision. =A0However, because of the dramatic reduction, I also would like t=
o make sure it is absolutely correct before submitting it - and also get a =
sense of why there was such a difference. -- I know cassandra 1.0 does data=
 compression, but does the schema change from supercolumn to regular column=
 also help reduce storage usage? =A0Thanks.</div>

<div><br></div><div>-- Y.</div>

--f46d044304427991bf04bc53a083--