incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Luciani <jak...@gmail.com>
Subject Re: Is the secondary index re-built under compaction?
Date Tue, 17 Apr 2012 14:20:59 GMT
How many indexes are there?

On Tue, Apr 17, 2012 at 10:16 AM, Maxim Potekhin <potekhin@bnl.gov> wrote:

>  Yes. Sorry I didn't mention this, but of course I'm checking on indexes
> once in a while.
> So yes, they are marked as built.
>
> All of this started happening after a few days of continuous loading
> process. Since
> the nodes have good hardware (24 cores + SSD), the apparent load on each
> node
> was nothing remarkable, even at 20kHz insertion rate. But maybe I'm being
> overoptimistic.
>
> Maxim
>
>
>
> On 4/17/2012 10:12 AM, Jake Luciani wrote:
>
> Hmm that does sound fishy.
>
>  When you run show keyspaces from cassandra-cli it shows which indexes
> are built.  Are they marked built in your column family?
>
>  -Jake
>
>  On Tue, Apr 17, 2012 at 10:09 AM, Maxim Potekhin <potekhin@bnl.gov>wrote:
>
>>  I understand that indexes are CFs. But the compaction stats says it's
>> building the
>> index, not compacting the corresponding CF. Either that's an ambiguous
>> diagnostic,
>> or indeed something is not right with my rig as of late.
>>
>> Maxim
>>
>>
>>
>>
>> On 4/17/2012 10:05 AM, Jake Luciani wrote:
>>
>> Well, the since the secondary indexes are themselves column families they
>> too are compacted along with everything else.
>>
>> On Tue, Apr 17, 2012 at 10:02 AM, Maxim Potekhin <potekhin@bnl.gov>wrote:
>>
>>>  Thanks Jake. Then I am definitely seeing weirdness, as there are tons of
>>> "pending tasks" in compaction stats, and tons of index files created in
>>> the
>>> data directory. Plus it does tell me that it is building the secondary
>>> index,
>>> and that seems to be happening at an amazingly glacial pace.
>>>
>>> I have 2 CFs there, with multiple secondary indexes. I'll try
>>> to compact the CF one by one, reboot and see if that helps.
>>>
>>> Maxim
>>>
>>>
>>>
>>> On 4/17/2012 9:53 AM, Jake Luciani wrote:
>>>
>>> No, the indexes are not rebuilt every compaction.  Only if you manually
>>> rebuild or bootstrap a new node does it use compaction manager to rebuild.
>>>
>>> On Tue, Apr 17, 2012 at 9:47 AM, Maxim Potekhin <potekhin@bnl.gov>wrote:
>>>
>>>>  Thanks Aaaron. Just to be clear, every time I do a compaction,
>>>> I rebuild all indexes from scratch. Right?
>>>>
>>>> Maxim
>>>>
>>>>
>>>>
>>>> On 4/17/2012 6:16 AM, aaron morton wrote:
>>>>
>>>> Yes secondary index builds are done via the compaction manager.
>>>>
>>>>  Cheers
>>>>
>>>>     -----------------
>>>> Aaron Morton
>>>> Freelance Developer
>>>> @aaronmorton
>>>> http://www.thelastpickle.com
>>>>
>>>>  On 17/04/2012, at 1:06 PM, Maxim Potekhin wrote:
>>>>
>>>>  I noticed that "nodetool compactionstats" shows the building of the
>>>> secondary index while
>>>> I initiate compaction. Is this to be expected? Cassandra version 0.8.8.
>>>>
>>>> Thank you
>>>>
>>>> Maxim
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>  --
>>> http://twitter.com/tjake
>>>
>>>
>>>
>>
>>
>>  --
>> http://twitter.com/tjake
>>
>>
>>
>
>
>  --
> http://twitter.com/tjake
>
>
>


-- 
http://twitter.com/tjake

Mime
View raw message