incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxim Potekhin <potek...@bnl.gov>
Subject Re: Is the secondary index re-built under compaction?
Date Tue, 17 Apr 2012 14:16:58 GMT
Yes. Sorry I didn't mention this, but of course I'm checking on indexes 
once in a while.
So yes, they are marked as built.

All of this started happening after a few days of continuous loading 
process. Since
the nodes have good hardware (24 cores + SSD), the apparent load on each 
node
was nothing remarkable, even at 20kHz insertion rate. But maybe I'm 
being overoptimistic.

Maxim


On 4/17/2012 10:12 AM, Jake Luciani wrote:
> Hmm that does sound fishy.
>
> When you run show keyspaces from cassandra-cli it shows which indexes 
> are built.  Are they marked built in your column family?
>
> -Jake
>
> On Tue, Apr 17, 2012 at 10:09 AM, Maxim Potekhin <potekhin@bnl.gov 
> <mailto:potekhin@bnl.gov>> wrote:
>
>     I understand that indexes are CFs. But the compaction stats says
>     it's building the
>     index, not compacting the corresponding CF. Either that's an
>     ambiguous diagnostic,
>     or indeed something is not right with my rig as of late.
>
>     Maxim
>
>
>
>
>     On 4/17/2012 10:05 AM, Jake Luciani wrote:
>>     Well, the since the secondary indexes are themselves
>>     column families they too are compacted along with everything else.
>>
>>     On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin
>>     <potekhin@bnl.gov <mailto:potekhin@bnl.gov>> wrote:
>>
>>         Thanks Jake. Then I am definitely seeing weirdness, as there
>>         are tons of
>>         "pending tasks" in compaction stats, and tons of index files
>>         created in the
>>         data directory. Plus it does tell me that it is building the
>>         secondary index,
>>         and that seems to be happening at an amazingly glacial pace.
>>
>>         I have 2 CFs there, with multiple secondary indexes. I'll try
>>         to compact the CF one by one, reboot and see if that helps.
>>
>>         Maxim
>>
>>
>>
>>         On 4/17/2012 9:53 AM, Jake Luciani wrote:
>>>         No, the indexes are not rebuilt every compaction.  Only if
>>>         you manually rebuild or bootstrap a new node does it use
>>>         compaction manager to rebuild.
>>>
>>>         On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin
>>>         <potekhin@bnl.gov <mailto:potekhin@bnl.gov>> wrote:
>>>
>>>             Thanks Aaaron. Just to be clear, every time I do a
>>>             compaction,
>>>             I rebuild all indexes from scratch. Right?
>>>
>>>             Maxim
>>>
>>>
>>>
>>>             On 4/17/2012 6:16 AM, aaron morton wrote:
>>>>             Yes secondary index builds are done via the compaction
>>>>             manager.
>>>>
>>>>             Cheers
>>>>
>>>>             -----------------
>>>>             Aaron Morton
>>>>             Freelance Developer
>>>>             @aaronmorton
>>>>             http://www.thelastpickle.com
>>>>
>>>>             On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote:
>>>>
>>>>>             I noticed that "nodetool compactionstats" shows the
>>>>>             building of the secondary index while
>>>>>             I initiate compaction. Is this to be expected?
>>>>>             Cassandra version 0.8.8.
>>>>>
>>>>>             Thank you
>>>>>
>>>>>             Maxim
>>>>>
>>>>
>>>
>>>
>>>
>>>
>>>         -- 
>>>         http://twitter.com/tjake
>>
>>
>>
>>
>>     -- 
>>     http://twitter.com/tjake
>
>
>
>
> -- 
> http://twitter.com/tjake


Mime
View raw message