incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Leeds <randall.le...@gmail.com>
Subject Re: Preserving seq order through replication
Date Fri, 05 Mar 2010 19:13:26 GMT
I believe replication right now sorts the incoming documents by seq
(comment says smth like 'just in case'), but then they are fetched
with some amount of concurrency and inserted as they arrive. Adam,
please chime in if I'm reading it wrong, as I think some of those
comments are yours.

On Fri, Mar 5, 2010 at 09:10, Adam Kocoloski <kocolosk@apache.org> wrote:
> With that said, making replication preserve the update order is probably not very difficult
or expensive to do.  Best,
>
> Adam

It could be a performance loss if couch_rep_writer had to buffer
writes to preserve insertion order. Alternatively, to prevent the
write queue from growing unboundedly if one document repeatedly fails,
couch_rep_reader could wait for a chunk of contiguous documents before
handing them over to the writer.

Concurrently fetching documents rather than just pipelining them on
one http connection might not seem beneficial at first glance, but a
source which has disks that can service concurrent reads stands to
benefit. When the source can't actually do this it's up to the
OS/FS/Disk to order reads in a way that is as optimal as possible.

Randall

Mime
View raw message