Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of paul.joseph.davis@gmail.com
 designates 209.85.214.175 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <08224CCB-5CDF-46D5-9678-F2C59516AE4E@apache.org>
References: 
 <CAOxguM0LspwwGbBLntVu5rZgnK9qsuk8o6XcyqnwnBy3Y+S-yA@mail.gmail.com>
 <4AD73424-F7BB-49F4-ABC9-43A88D34BDCA@couchbase.com>
 <CACLy94UmJDqDg3JjHisUxJLPtJoZxp9MyxdJvaNW+p67rCp5wQ@mail.gmail.com>
 <CAK0S_BDS1-Z7mHRHF6vD0De77iB5z-u4oQt+SQgkormSFZMjbA@mail.gmail.com>
 <DB2C717B-928D-4EF9-B9E2-D872EA725041@couchbase.com>
 <CACLy94VHX6GqQv_2v1ZgY674WoV8PzLLQSWz1_vV+7XvGUyVdQ@mail.gmail.com>
 <CAOxguM1wchf13xMMOfTr8hWJQB-=WmU_NvF4Nqhjspt_UkaDpA@mail.gmail.com>
 <BC203F6D-144E-48A8-AD04-E126C77203C4@couchbase.com>
 <CAOxguM0MhKk8sMRLY0HjTfymfYnuB6TrwaA789H1WryPmQFhbw@mail.gmail.com>
 <82D0792E-80AA-452A-9408-9FA648514B7F@couchbase.com>
 <0FE12D96-ECA4-4D29-A3F1-235A22E35BBB@apache.org>
 <CAOxguM0MPHyJ1dPhbbvo2ZkUWfswObw=pYteEp9PHW-nTQVNyw@mail.gmail.com>
 <08224CCB-5CDF-46D5-9678-F2C59516AE4E@apache.org>
From: Paul Davis <paul.joseph.davis@gmail.com>
Date: Mon, 17 Feb 2014 13:04:23 -0600
Message-ID: 
 <CAJ_m3YC6JYKqC78VGWdXN_Ler+u+b-PgegXhPPtg5BHS2qM_4A@mail.gmail.com>
Subject: Re: Replication vs. Compaction
To: "user@couchdb.apache.org" <user@couchdb.apache.org>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Typo'ed:

"so it's unbalancing as much"

should read:

"so it's *not* unbalancing as much"

On Mon, Feb 17, 2014 at 12:49 PM, Robert Samuel Newson
<rnewson@apache.org> wrote:
> Replication will not rebalance the tree, no. It's just adding to the end =
(and unbalancing the tree).
>
> The updates are happening in batches, though, so it's unbalancing as much=
 as the original individual updates did.
>
> B.
>
> On 17 Feb 2014, at 16:16, Boaz Citrin <bcitrin@gmail.com> wrote:
>
>> Did some more testing. seems like indeed compaction is faster than repli=
cation.
>>
>> One thing I observe though is that replication doesn't result the same
>> as compaction;
>> While it only copies the leaves, I suspect it doesn't produce a
>> balanced tree, so subsequent compaction is needed anyway (and indeed
>> cuts the file size big time).
>>
>> Am I wrong here?
>>
>> On Mon, Feb 3, 2014 at 8:28 PM, Adam Kocoloski <kocolosk@apache.org> wro=
te:
>>> On Jan 31, 2014, at 3:43 PM, Jens Alfke <jens@couchbase.com> wrote:
>>>
>>>> On Jan 31, 2014, at 12:07 PM, Boaz Citrin <bcitrin@gmail.com> wrote:
>>>>
>>>>> But if replication only copies the leaf then it makes sense that it i=
s
>>>>> fatser, at least on the same machine. Instead of balancing a tree it =
just
>>>>> copies a single revision.
>>>>
>>>> Um, no. The copied revision has to be inserted into the tree on the ta=
rget database. Worse, the target database is assumed to be 'live' during th=
e whole process, so its tree can't be updated as efficiently as during a re=
plication, where the new database file isn't going to be used at all until =
the whole procedure finishes.
>>>>
>>>> Sorry to pull rank, but while I haven't worked on CouchDB itself, I've=
 written 1 1/2 CouchDB-compatible replicators, and I've worked on a C-based=
 compactor for CouchDB-format b-tree files. I'm pretty sure that compaction=
 is a lot faster. There's just much less work that it has to do.
>>>>
>>>> I agree with Jason that you probably need a faster server (or disk) th=
at will let you compact effectively.
>>>>
>>>> --Jens
>>>
>>> Agreed, and also worth pointing out that we've developed a compactor th=
at is far more efficient than the one in master.  It uses less I/O and gene=
rates a smaller file to boot:
>>>
>>> https://git-wip-us.apache.org/repos/asf?p=3Dcouchdb-couch.git;a=3Dcommi=
t;h=3D5d3753d0662cfa676fdf65d0a543be205499ec11
>>>
>>> Hopefully we can land it soon.  Regards,
>>>
>>> Adam
>