couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Newson <robert.new...@gmail.com>
Subject Re: Newbie question: compaction and mvcc consistency?
Date Wed, 26 May 2010 20:49:48 GMT
The potential inability to complete compaction in write-saturated
environments is captured in
http://issues.apache.org/jira/browse/COUCHDB-487 with a patch.

I think kocolosk has recently written a patch that improves the way
that data needs to be read to perform compaction, which in turn
reduces the likelihood of not flipping over.

There's also the @dev thread on using a sequence of files (like JE)
instead of a single one, which also reduces the problem.

B.

On Wed, May 26, 2010 at 9:35 PM, Randall Leeds <randall.leeds@gmail.com> wrote:
> On Wed, May 26, 2010 at 13:29, Robert Buck <buck.robert.j@gmail.com> wrote:
>> On Wed, May 26, 2010 at 3:00 PM, Randall Leeds <randall.leeds@gmail.com> wrote:
>>> The switch to the new, compacted database won't happen so long as
>>> there are references to the old one. (r) will not disappear until (i)
>>> is done with it.
>>
>> Curious, you said "switch to the new [database]". Does this imply that
>> compaction works by creating a new database file adjacent to the old
>> one?
>
> Yes.
>
>>
>> If this is what you are suggesting, I have another question... I also
>> read that compaction process may never catch up with the writes if
>> they never let up. So along this specific train of thought, does Couch
>> perform compaction by walking through the database in a forward-only
>> manner?
>
> If I understand correctly the answer is 'yes'. Meanwhile, new writes
> still hit the old database file as the compactor walks the old tree.
> If there are new changes when the compactor finishes it will walk the
> new changes starting from the root. Typically this process quickly
> gets faster and faster on busy databases until it catches up
> completely and the switch can be made.
>
> That said, you can construct an environment where compaction will
> never finish, but I haven't seen reports of it happening in the wild.
>

Mime
View raw message