Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of edlinuxguru@gmail.com
 designates 209.85.213.172 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CANAZdzW2=x75U95ZyfKmJdMhP7mex6OpgdK_d3RTyHqf9xG7jQ@mail.gmail.com>
References: 
 <CANAZdzVVFRjAjecwp8=mdUAeNfG+fG6c14qS1ERvkHy5nTVcMw@mail.gmail.com>
	<3BC67EDA-4C35-410A-94CF-CC49FD962114@gmail.com>
	<CANAZdzW2=x75U95ZyfKmJdMhP7mex6OpgdK_d3RTyHqf9xG7jQ@mail.gmail.com>
Date: Sat, 11 Aug 2012 14:31:37 -0400
Message-ID: 
 <CAENxBwxK5Q5yGJYpUQcF4DZzt-k-gvh-mi8Vyh_y1B-_8b-JqA@mail.gmail.com>
Subject: Re: quick question about data layout on disk
From: Edward Capriolo <edlinuxguru@gmail.com>
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Aaron,

I have not deep dived the data files in a while but this is how I understand it.

http://wiki.apache.org/cassandra/ArchitectureSSTable

There is no need to store the row key each time with the column.
RowKey to columns is a one to many relationship. This would be a
diagram of a physical file:

Hbase does it like this i guess (Or it used to I am not up on the news):

rowkey1,column1,value1
rowkey1,column2,value2

First, I believe they repeat the row key for each column, which is not
a huge deal because you should always use compression but it is a bit
wasteful especially for a non-compressed table.

I know this has some impact on very wide rows because a single rowkey
must fit inside this structure of an hfile. ? (again its been a while)

But to get back to your question. In cassandra:

sstable1
rowkey1: numberof columns 26
(column1,value1,ts1)
....
(column26,value26,ts26)

sstable2
rowkey1: numberof columns 1
(column1,value1,ts2)

The rowkey appears once in a given sstable if the row has 1 or more
columns in the sstable.

On the read path Cassandra searches all sstables find all the columns
for a row (bloom filters and other criteria eliminate some sstables
from read path). It then merges the row factoring in tombstones and
the last update win rules for a column.


On Sat, Aug 11, 2012 at 2:03 PM, Aaron Turner <synfinatic@gmail.com> wrote:
> So how does that work?  An sstable is for a single CF, but it can and
> likely will have multiple rows.  There is no read to write and as I
> understand it, writes are append operations.
>
> So if you have an sstable with say 26 different rows (A-Z) already in
> it with a bunch of columns and you add a new column to row J, how does
> Cassandra store the column/value pair on disk in a way to refer to row
> J without re-writing the row key or some representation of it?
>
> Thanks,
> Aaron
>
> On Fri, Aug 10, 2012 at 7:53 PM, Terje Marthinussen
> <tmarthinussen@gmail.com> wrote:
>> Rowkey is stored only once in any sstable file.
>>
>> That is, in the spesial case where you get sstable file per column/value, you are correct, but normally, I guess most of us are storing more per key.
>>
>> Regards,
>> Terje
>>
>> On 11 Aug 2012, at 10:34, Aaron Turner <synfinatic@gmail.com> wrote:
>>
>>> Curious, but does cassandra store the rowkey along with every
>>> column/value pair on disk (pre-compaction) like Hbase does?  If so
>>> (which makes the most sense), I assume that's something that is
>>> optimized during compaction?
>>>
>>>
>>> --
>>> Aaron Turner
>>> http://synfin.net/         Twitter: @synfinatic
>>> http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Windows
>>> Those who would give up essential Liberty, to purchase a little temporary
>>> Safety, deserve neither Liberty nor Safety.
>>>    -- Benjamin Franklin
>>> "carpe diem quam minimum credula postero"
>
>
>
> --
> Aaron Turner
> http://synfin.net/         Twitter: @synfinatic
> http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Windows
> Those who would give up essential Liberty, to purchase a little temporary
> Safety, deserve neither Liberty nor Safety.
>     -- Benjamin Franklin
> "carpe diem quam minimum credula postero"