incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filipe David Manana <fdman...@apache.org>
Subject Re: Replication: stalled?
Date Tue, 01 Mar 2011 19:26:59 GMT
On Tue, Mar 1, 2011 at 6:54 PM, Wayne Conrad <wayne@databill.com> wrote:
> On 03/01/11 03:15, Filipe David Manana wrote:
>>
>> You can also try the new replicator that landed recently into trunk
>> (from where 1.2 will be cut from), it adds more parallelism so you'll
>> likely see a better performance:
>>
>> http://s.apache.org/KsY
>>
>>
>> https://github.com/apache/couchdb/commit/34eb4175f8547546cd76fbeb006421020cbf0d71
>
> Version 1.2.0ac052866-git replicates my ginormous documents with panache.
>  My test-case-de-jeur is a document with 32,700 attachments, 4k each.
>  Inserting 100 attachments at a time, the document gets inserted into the
> source database in 42 seconds.  The destination database is pulling the
> changes so fast I can't measure the latency by refreshing futon: The moment
> I'm done adding the last attachment to the source database, it appears in
> the destination database.  Outstanding!
>
> I had only one small issue building it: on Debian, the package
> "erlang-eunit" is required to run "make check."  Is that worth mentioning in
> the DEVELOPER file?
>
> Before I put it in production, "make check" reports some failures.  Do these
> matter?

No they're related to the vhost and OS daemon features. The former was
fixed this morning I believe, while the later happens often but not
always. They're completely unrelated to the replicator, rest assured
:)

>
> Test Summary Report
> -------------------
> /var/lib/couchdb/custom-install-tree/couchdb/test/etap/160-vhosts.t
>    (Wstat: 0 Tests: 10 Failed: 0)
>  Parse errors: Bad plan.  You planned 14 tests but ran 10.
> /var/lib/couchdb/custom-install-tree/couchdb/test/etap/173-os-daemon-cfg-register.t
> (Wstat: 0 Tests: 8 Failed: 0)
>  Parse errors: Bad plan.  You planned 27 tests but ran 8.
> Files=36, Tests=724, 52 wallclock secs ( 0.15 usr  0.04 sys + 11.29 cusr
>  1.16 csys = 12.64 CPU)
> Result: FAIL
>
> Oh, and: Can I try this out by just using this version on the destination
> database, or does the source database also need need version?
>

This new replicator should work flawlessly with CouchDB 1.0.2+. Pull
replicating (when there are compressed attachments in the source) from
1.0.1 (or bellow) server might not work under some circumstances -
this was fixed in 1.0.2 (a fix to the document multipart/mixed and
multipart/related streaming APIs).

Glad to know it's working much better for you compared to the old replicator :)

regards,

-- 
Filipe David Manana,
fdmanana@gmail.com, fdmanana@apache.org

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Mime
View raw message