couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From till <klimp...@gmail.com>
Subject Re: _compact on 0.10.0 <> availability
Date Tue, 13 Apr 2010 17:28:08 GMT
On Tue, Apr 13, 2010 at 6:39 PM, J Chris Anderson <jchris@gmail.com> wrote:
>
> On Apr 13, 2010, at 9:31 AM, till wrote:
>
>> Hey devs,
>>
>> I'm trying to compact a production database here (in hope to recover
>> some space), and made the following observations:
>>
>> * the set is 212+ million docs
>> * currently 0.8 TB in size
>> * the instance (XL) has 2 cores, one is idle, the other maybe utilized at 10%
>> * memory - 2 of 15 GB taken, no spikes
>> * io - well it's EBS :(
>>
>> When I started _compact read operations slowed down (I'll give you 20
>> Mississippi's for something that loads instantly otherwise).
>> Everything "eventually" worked, but it slowed down tremendously.
>>
>> I restarted the CouchDB process and everything is back to "snap".
>>
>> Does anyone have any insight on why that is the case?
>
> I'm guessing this is an EBS / EC2 issue. You are probably saturating the IO pipeline.
It's too bad there's not an easy way to 'nice' the compaction IO.
>
> If you got unlucky and are on a particularly bad EBS / EC2 instance, you might do best
to start up a new Couch in the same availability zone and replicate across to it. This will
accomplish more-or-less the same effect as compaction.

This wouldn't help either, plus it would take roughly 60 days to complete. :D

With some help of Adam, we figured out earlier this week that this is the case.

The replicator maxes out at ~30 docs/sec which results in more (!) or
less uncompacted storage on the target node as well. When we compacted
the target we ended up gaining more space (again) -- if I remember
correctly we went from 2 GB to 97 MB.

Till

>
>>
>> Till
>
>

Mime
View raw message