Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of tuxracer69@gmail.com
 designates 74.125.82.44 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject
         :content-type:content-transfer-encoding;
        b=EnXTEF05CPHmULxZmRHvA+O8Lze8hnsllG5eVI4bZhSQfXT9q6/40iVLLfIX4FawVC
         kiHyCUxC7a1UNUYqv8n4LlJo85/QSeBLke8r9OPCPqj3/+nVA1+tv+CNWWIVr06rGpu2
         nzra8Om3YTHW2UTUWS3C7fUx197IiSIDp0DWM=
Message-ID: <4BD473DF.1020406@gmail.com>
Date: Sun, 25 Apr 2010 17:54:55 +0100
From: TuX RaceR <tuxracer69@gmail.com>
User-Agent: Mozilla-Thunderbird 2.0.0.22 (X11/20090701)
MIME-Version: 1.0
To: user@cassandra.apache.org
Subject: newbie question on how columns names are indexed/lucene limitations?
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hello Cassandra Users,

When use the RandomPartinionner and a simple ColumnFamily/Columns (i.e. 
no SuperColumns) my understanding is that one signle Row can store 
millions of columns.

If I look at the http://wiki.apache.org/cassandra/API, I understand that 
I can get a subset of the millions of columns defined above using:
SlicePredicate->ColumnNames or SlicePredicate->SliceRange

My question is about the implementation of this columns 'selection'.
I vaguely remember reading somewhere (but I cannot find the link again) 
that this was implemented using a Lucene index over the column names for 
each row.
Is that true? Is there a small lucene index per row?

Also we know from that lucene have some limitations 
http://lucene.apache.org/java/3_0_1/fileformats.html#Limitations : you 
cannot index more than 2.1 billions documents as a document ID is mapped 
to a 32 bits int.

As I plan to store in column names the ID of my cassandra documents (the 
global number of documents can go well beyond 2.1 billions), will I be 
hit by the lucene limitations? I.e can I store cassandra documents ID 
(i.e keys) in column names, if in each individual row there are no more 
than few millions of those IDs? I guess the answer is "yes I can", 
because lucandra uses a similar schema but it is not clear for me why. 
Is that because the lucene index is made on each row and what really 
matters in the number of columns in one single row and not the number of 
distinct column names (globally over all the rows)?


Thanks in advance
TuX