Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of potekhin@bnl.gov designates
 130.199.3.132 as permitted sender)
Message-ID: <4EC0769D.6090402@bnl.gov>
Date: Sun, 13 Nov 2011 21:02:05 -0500
From: Maxim Potekhin <potekhin@bnl.gov>
User-Agent: Mozilla/5.0 (Windows NT 6.0;
 rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1
MIME-Version: 1.0
To: user@cassandra.apache.org
Subject: Re: Mass deletion -- slowing down
References: <1302618388.3794.34.camel@mierdi-laptop>
 <4DA47CCD.50509@panasiangroup.com> <1302630144.1732.2.camel@Avalon>
 <4DA491EE.6010500@panasiangroup.com> <4EB84C9F.8040208@bnl.gov>
 <CAKkz8Q03bk1=moG__noMBV6zUAo+9i7ocLMQesWceUU9voc6=g@mail.gmail.com>
 <4EBC7AB8.7030105@bnl.gov>
 <CAO5xsd2DKqz7dv1zpp3G3g3wvquu5YiuD7oNhzjWUNqwjvCRgw@mail.gmail.com>
 <4EC066E7.9090707@bnl.gov>
 <CAAafH9RCwXbmd_2wYiwrR686Y4gmzewTf-5ZEzc_6NFCGxC9wA@mail.gmail.com>
 <4EC06E1B.9020905@bnl.gov>
 <CAO5xsd3fS6yAhBzFrZdEMQ7wCpveneBMrM0vH4V9jOKUvxzBFw@mail.gmail.com>
In-Reply-To: 
 <CAO5xsd3fS6yAhBzFrZdEMQ7wCpveneBMrM0vH4V9jOKUvxzBFw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Thanks Peter,

I'm not sure I entirely follow. By the oldest data, do you mean the
primary key corresponding to the limit of the time horizon? Unfortunately,
unique IDs and the timstamps do not correlate in the sense that 
chronologically
"newer" entries might have a smaller sequential ID. That's because the 
timestamp
corresponds to the last update that's stochastic in the sense that the 
jobs can take
from seconds to days to complete. As I said I'm not sure I understood you
correctly.

Also, I note that queries on different dates (i.e. not "contaminated" 
with lots
of tombstones) work just fine, which is consistent with the picture that
emerged so far.

Theoretically -- would compaction or cleanup help?

Thanks

Maxim


On 11/13/2011 8:39 PM, Peter Schuller wrote:
>> I do limit the number of rows I'm asking for in Pycassa. Queries on primary
>> keys still work fine,
> Is it feasable in your situation to keep track of the oldest possible
> data (for example, if there is a single sequential writer that rotates
> old entries away it could keep a record of what the oldest might be)
> so that you can bound your index lookup>= that value (and avoid the
> tombstones)?
>