incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Coverston <ben.covers...@datastax.com>
Subject Re: Is the secondary index re-built under compaction?
Date Tue, 17 Apr 2012 21:45:50 GMT
Every time a repair creates a new SSTable segment you'll get another index
build compaction for the keys in that segment.

On Tue, Apr 17, 2012 at 8:32 AM, Maxim Potekhin <potekhin@bnl.gov> wrote:

>  The "offending" CF only has one. The other one, that seems to behave well,
> has nine.
>
> Maxim
>
>
>
> On 4/17/2012 10:20 AM, Jake Luciani wrote:
>
> How many indexes are there?
>
> On Tue, Apr 17, 2012 at 10:16 AM, Maxim Potekhin <potekhin@bnl.gov> wrote:
>
>>  Yes. Sorry I didn't mention this, but of course I'm checking on indexes
>> once in a while.
>> So yes, they are marked as built.
>>
>> All of this started happening after a few days of continuous loading
>> process. Since
>> the nodes have good hardware (24 cores + SSD), the apparent load on each
>> node
>> was nothing remarkable, even at 20kHz insertion rate. But maybe I'm being
>> overoptimistic.
>>
>> Maxim
>>
>>
>>
>> On 4/17/2012 10:12 AM, Jake Luciani wrote:
>>
>> Hmm that does sound fishy.
>>
>>  When you run show keyspaces from cassandra-cli it shows which indexes
>> are built.  Are they marked built in your column family?
>>
>>  -Jake
>>
>>  On Tue, Apr 17, 2012 at 10:09 AM, Maxim Potekhin <potekhin@bnl.gov>wrote:
>>
>>>  I understand that indexes are CFs. But the compaction stats says it's
>>> building the
>>> index, not compacting the corresponding CF. Either that's an ambiguous
>>> diagnostic,
>>> or indeed something is not right with my rig as of late.
>>>
>>> Maxim
>>>
>>>
>>>
>>>
>>> On 4/17/2012 10:05 AM, Jake Luciani wrote:
>>>
>>> Well, the since the secondary indexes are themselves
>>> column families they too are compacted along with everything else.
>>>
>>> On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin <potekhin@bnl.gov>wrote:
>>>
>>>>  Thanks Jake. Then I am definitely seeing weirdness, as there are tons
>>>> of
>>>> "pending tasks" in compaction stats, and tons of index files created in
>>>> the
>>>> data directory. Plus it does tell me that it is building the secondary
>>>> index,
>>>> and that seems to be happening at an amazingly glacial pace.
>>>>
>>>> I have 2 CFs there, with multiple secondary indexes. I'll try
>>>> to compact the CF one by one, reboot and see if that helps.
>>>>
>>>> Maxim
>>>>
>>>>
>>>>
>>>> On 4/17/2012 9:53 AM, Jake Luciani wrote:
>>>>
>>>> No, the indexes are not rebuilt every compaction.  Only if you manually
>>>> rebuild or bootstrap a new node does it use compaction manager to rebuild.
>>>>
>>>> On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin <potekhin@bnl.gov>wrote:
>>>>
>>>>>  Thanks Aaaron. Just to be clear, every time I do a compaction,
>>>>> I rebuild all indexes from scratch. Right?
>>>>>
>>>>> Maxim
>>>>>
>>>>>
>>>>>
>>>>> On 4/17/2012 6:16 AM, aaron morton wrote:
>>>>>
>>>>> Yes secondary index builds are done via the compaction manager.
>>>>>
>>>>>  Cheers
>>>>>
>>>>>     -----------------
>>>>> Aaron Morton
>>>>> Freelance Developer
>>>>> @aaronmorton
>>>>> http://www.thelastpickle.com
>>>>>
>>>>>  On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote:
>>>>>
>>>>>  I noticed that "nodetool compactionstats" shows the building of the
>>>>> secondary index while
>>>>> I initiate compaction. Is this to be expected? Cassandra version 0.8.8.
>>>>>
>>>>> Thank you
>>>>>
>>>>> Maxim
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>  --
>>>> http://twitter.com/tjake
>>>>
>>>>
>>>>
>>>
>>>
>>>  --
>>> http://twitter.com/tjake
>>>
>>>
>>>
>>
>>
>>  --
>> http://twitter.com/tjake
>>
>>
>>
>
>
>  --
> http://twitter.com/tjake
>
>
>


-- 
Ben Coverston
DataStax -- The Apache Cassandra Company

Mime
View raw message