Return-Path: Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: (qmail 43451 invoked from network); 5 Mar 2010 19:27:48 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 5 Mar 2010 19:27:48 -0000 Received: (qmail 88615 invoked by uid 500); 5 Mar 2010 19:27:33 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 88558 invoked by uid 500); 5 Mar 2010 19:27:33 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 88550 invoked by uid 99); 5 Mar 2010 19:27:33 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Mar 2010 19:27:33 +0000 X-ASF-Spam-Status: No, hits=3.6 required=10.0 tests=FREEMAIL_FROM,FS_REPLICA,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of adam.kocoloski@gmail.com designates 72.14.220.152 as permitted sender) Received: from [72.14.220.152] (HELO fg-out-1718.google.com) (72.14.220.152) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Mar 2010 19:27:24 +0000 Received: by fg-out-1718.google.com with SMTP id 22so324806fge.5 for ; Fri, 05 Mar 2010 11:27:04 -0800 (PST) Received: by 10.87.69.17 with SMTP id w17mr1821619fgk.41.1267817224557; Fri, 05 Mar 2010 11:27:04 -0800 (PST) Received: from [10.1.10.164] (c-66-31-20-188.hsd1.ma.comcast.net [66.31.20.188]) by mx.google.com with ESMTPS id d6sm3025998fga.17.2010.03.05.11.27.03 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 05 Mar 2010 11:27:04 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1077) Subject: Re: Preserving seq order through replication From: Adam Kocoloski In-Reply-To: Date: Fri, 5 Mar 2010 14:27:03 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: dev@couchdb.apache.org X-Mailer: Apple Mail (2.1077) X-Virus-Checked: Checked by ClamAV on apache.org On Mar 5, 2010, at 2:13 PM, Randall Leeds wrote: > I believe replication right now sorts the incoming documents by seq > (comment says smth like 'just in case'), but then they are fetched > with some amount of concurrency and inserted as they arrive. Adam, > please chime in if I'm reading it wrong, as I think some of those > comments are yours. Yep, that's basically it. > On Fri, Mar 5, 2010 at 09:10, Adam Kocoloski = wrote: >> With that said, making replication preserve the update order is = probably not very difficult or expensive to do. Best, >>=20 >> Adam >=20 > It could be a performance loss if couch_rep_writer had to buffer > writes to preserve insertion order. Alternatively, to prevent the > write queue from growing unboundedly if one document repeatedly fails, > couch_rep_reader could wait for a chunk of contiguous documents before > handing them over to the writer. If we were to do this, I'd implement it that 2nd way, where the reader = only hands over contiguous blocks to the writer, and doesn't "get too = far ahead of itself" > Concurrently fetching documents rather than just pipelining them on > one http connection might not seem beneficial at first glance, but a > source which has disks that can service concurrent reads stands to > benefit. When the source can't actually do this it's up to the > OS/FS/Disk to order reads in a way that is as optimal as possible. +1 Adam=