couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <>
Subject Re: Replication vs. Compaction
Date Mon, 17 Feb 2014 19:04:23 GMT

"so it's unbalancing as much"

should read:

"so it's *not* unbalancing as much"

On Mon, Feb 17, 2014 at 12:49 PM, Robert Samuel Newson
<> wrote:
> Replication will not rebalance the tree, no. It's just adding to the end (and unbalancing
the tree).
> The updates are happening in batches, though, so it's unbalancing as much as the original
individual updates did.
> B.
> On 17 Feb 2014, at 16:16, Boaz Citrin <> wrote:
>> Did some more testing. seems like indeed compaction is faster than replication.
>> One thing I observe though is that replication doesn't result the same
>> as compaction;
>> While it only copies the leaves, I suspect it doesn't produce a
>> balanced tree, so subsequent compaction is needed anyway (and indeed
>> cuts the file size big time).
>> Am I wrong here?
>> On Mon, Feb 3, 2014 at 8:28 PM, Adam Kocoloski <> wrote:
>>> On Jan 31, 2014, at 3:43 PM, Jens Alfke <> wrote:
>>>> On Jan 31, 2014, at 12:07 PM, Boaz Citrin <> wrote:
>>>>> But if replication only copies the leaf then it makes sense that it is
>>>>> fatser, at least on the same machine. Instead of balancing a tree it
>>>>> copies a single revision.
>>>> Um, no. The copied revision has to be inserted into the tree on the target
database. Worse, the target database is assumed to be 'live' during the whole process, so
its tree can't be updated as efficiently as during a replication, where the new database file
isn't going to be used at all until the whole procedure finishes.
>>>> Sorry to pull rank, but while I haven't worked on CouchDB itself, I've written
1 1/2 CouchDB-compatible replicators, and I've worked on a C-based compactor for CouchDB-format
b-tree files. I'm pretty sure that compaction is a lot faster. There's just much less work
that it has to do.
>>>> I agree with Jason that you probably need a faster server (or disk) that
will let you compact effectively.
>>>> --Jens
>>> Agreed, and also worth pointing out that we've developed a compactor that is
far more efficient than the one in master.  It uses less I/O and generates a smaller file
to boot:
>>> Hopefully we can land it soon.  Regards,
>>> Adam

View raw message