incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxim Potekhin <potek...@bnl.gov>
Subject Re: Is the secondary index re-built under compaction?
Date Tue, 17 Apr 2012 14:32:22 GMT
The "offending" CF only has one. The other one, that seems to behave well,
has nine.

Maxim


On 4/17/2012 10:20 AM, Jake Luciani wrote:
> How many indexes are there?
>
> On Tue, Apr 17, 2012 at 10:16 AM, Maxim Potekhin <potekhin@bnl.gov 
> <mailto:potekhin@bnl.gov>> wrote:
>
>     Yes. Sorry I didn't mention this, but of course I'm checking on
>     indexes once in a while.
>     So yes, they are marked as built.
>
>     All of this started happening after a few days of continuous
>     loading process. Since
>     the nodes have good hardware (24 cores + SSD), the apparent load
>     on each node
>     was nothing remarkable, even at 20kHz insertion rate. But maybe
>     I'm being overoptimistic.
>
>     Maxim
>
>
>
>     On 4/17/2012 10:12 AM, Jake Luciani wrote:
>>     Hmm that does sound fishy.
>>
>>     When you run show keyspaces from cassandra-cli it shows which
>>     indexes are built.  Are they marked built in your column family?
>>
>>     -Jake
>>
>>     On Tue, Apr 17, 2012 at 10:09 AM, Maxim Potekhin
>>     <potekhin@bnl.gov <mailto:potekhin@bnl.gov>> wrote:
>>
>>         I understand that indexes are CFs. But the compaction stats
>>         says it's building the
>>         index, not compacting the corresponding CF. Either that's an
>>         ambiguous diagnostic,
>>         or indeed something is not right with my rig as of late.
>>
>>         Maxim
>>
>>
>>
>>
>>         On 4/17/2012 10:05 AM, Jake Luciani wrote:
>>>         Well, the since the secondary indexes are themselves
>>>         column families they too are compacted along with everything
>>>         else.
>>>
>>>         On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin
>>>         <potekhin@bnl.gov <mailto:potekhin@bnl.gov>> wrote:
>>>
>>>             Thanks Jake. Then I am definitely seeing weirdness, as
>>>             there are tons of
>>>             "pending tasks" in compaction stats, and tons of index
>>>             files created in the
>>>             data directory. Plus it does tell me that it is building
>>>             the secondary index,
>>>             and that seems to be happening at an amazingly glacial pace.
>>>
>>>             I have 2 CFs there, with multiple secondary indexes.
>>>             I'll try
>>>             to compact the CF one by one, reboot and see if that helps.
>>>
>>>             Maxim
>>>
>>>
>>>
>>>             On 4/17/2012 9:53 AM, Jake Luciani wrote:
>>>>             No, the indexes are not rebuilt every compaction.  Only
>>>>             if you manually rebuild or bootstrap a new node does it
>>>>             use compaction manager to rebuild.
>>>>
>>>>             On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin
>>>>             <potekhin@bnl.gov <mailto:potekhin@bnl.gov>> wrote:
>>>>
>>>>                 Thanks Aaaron. Just to be clear, every time I do a
>>>>                 compaction,
>>>>                 I rebuild all indexes from scratch. Right?
>>>>
>>>>                 Maxim
>>>>
>>>>
>>>>
>>>>                 On 4/17/2012 6:16 AM, aaron morton wrote:
>>>>>                 Yes secondary index builds are done via the
>>>>>                 compaction manager.
>>>>>
>>>>>                 Cheers
>>>>>
>>>>>                 -----------------
>>>>>                 Aaron Morton
>>>>>                 Freelance Developer
>>>>>                 @aaronmorton
>>>>>                 http://www.thelastpickle.com
>>>>>
>>>>>                 On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote:
>>>>>
>>>>>>                 I noticed that "nodetool compactionstats" shows
>>>>>>                 the building of the secondary index while
>>>>>>                 I initiate compaction. Is this to be expected?
>>>>>>                 Cassandra version 0.8.8.
>>>>>>
>>>>>>                 Thank you
>>>>>>
>>>>>>                 Maxim
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>             -- 
>>>>             http://twitter.com/tjake
>>>
>>>
>>>
>>>
>>>         -- 
>>>         http://twitter.com/tjake
>>
>>
>>
>>
>>     -- 
>>     http://twitter.com/tjake
>
>
>
>
> -- 
> http://twitter.com/tjake


Mime
View raw message