From user-return-4113-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Fri Mar 20 13:27:18 2009 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 12085 invoked from network); 20 Mar 2009 13:27:18 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 20 Mar 2009 13:27:18 -0000 Received: (qmail 93033 invoked by uid 500); 20 Mar 2009 13:27:15 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 92994 invoked by uid 500); 20 Mar 2009 13:27:15 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 92983 invoked by uid 99); 20 Mar 2009 13:27:15 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Mar 2009 06:27:14 -0700 X-ASF-Spam-Status: No, hits=2.7 required=10.0 tests=FS_REPLICA,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of adam.kocoloski@gmail.com designates 74.125.92.26 as permitted sender) Received: from [74.125.92.26] (HELO qw-out-2122.google.com) (74.125.92.26) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Mar 2009 13:27:07 +0000 Received: by qw-out-2122.google.com with SMTP id 8so533140qwh.29 for ; Fri, 20 Mar 2009 06:26:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=8vY4qY7zI0jj2SnkHzbRb0hsVKmWahzy82hVdOFeGgk=; b=wz+h3KZqPpnWWVFMURfB9f7zM8D/FZ148MJZQYoYgs7C4gtnj7yWF+sMmqkjBtmAuq 7t7w1am0QkxzfEc7HzJtGjP2LbHpVU8AC+YlrUYVT9oZXCa1LMkc4bi6fr7FCfE5CF7G LIrfc90eE+/XAMjz8B0ImCy+CE5VoUZILojdM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=TaRfdGgQW9E8z863d37kllI7S4A43Bj/2RD9oVP+fKs1TvP/8y3+mlY6qAdxmuHqJb MkVmef28zt03CIhP41pP6qn3TnkaBF1K6RWPVz36mW0NFQqIOGQjWjAoz9T537dXnSJ7 jIdOeg64s0QG6m2ZW9HIgbbYtp1Z1QR/yv8Bs= Received: by 10.224.10.205 with SMTP id q13mr5514710qaq.329.1237555606100; Fri, 20 Mar 2009 06:26:46 -0700 (PDT) Received: from COMPTON-THREE-TEN.MIT.EDU (COMPTON-THREE-TEN.MIT.EDU [18.109.6.55]) by mx.google.com with ESMTPS id 5sm1516976qwh.9.2009.03.20.06.26.45 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 20 Mar 2009 06:26:45 -0700 (PDT) Sender: Adam Kocoloski Message-Id: <752F15D9-45D7-4D5F-88BE-A59E127DCE32@apache.org> From: Adam Kocoloski To: user@couchdb.apache.org In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Subject: Re: Problem replicating from remote server Date: Fri, 20 Mar 2009 09:26:44 -0400 References: X-Mailer: Apple Mail (2.930.3) X-Virus-Checked: Checked by ClamAV on apache.org Hi Jaap, thanks. We did commit a recent change to the replicator that would make high-latency pull replications take significantly longer than before. Specifically, we stopped requesting multiple documents in parallel. This should only be a temporary situation; we wanted to make sure we had the replicator memory usage totally under control before turning the parallelization back on. I'm not always sure why the remote CouchDB server sometimes chooses to close the connection during replication. You're not the first person to see this happen, and in other cases these connection_closed errors have occurred when replicating inside a LAN. Did you know you can check the status of the replication in Futon? If you go to the "Status" page on the server where you initiated the replication you should see a replication task that updates every time it processes 100 docs on the source. Best, Adam On Mar 20, 2009, at 8:59 AM, Jaap van der Plas wrote: > Thanks, problem seems to be the connection: > > [Fri, 20 Mar 2009 12:45:47 GMT] [info] [<0.262.0>] retrying couch_rep > HTTP get request due to {error, connection_closed}: > http://localhost:5985/db/303ccbc3394aad84ff4bbce74470036b?revs=true&latest=true&open_revs= > ["4-3427878696"] > > [Fri, 20 Mar 2009 12:47:10 GMT] [info] [<0.262.0>] recording a > checkpoint at source update_seq 653 > > [Fri, 20 Mar 2009 12:47:11 GMT] [info] [<0.262.0>] retrying couch_rep > HTTP put request due to {error, connection_closed}: > http://localhost:5985/db/_local%2Ff7a03c6321dd9aa5652120b88cd4fb4d > > [Fri, 20 Mar 2009 12:47:12 GMT] [info] [<0.68.0>] 127.0.0.1 - - 'POST' > /_replicate 200 > > After this it did replicate correctly (for the first time.) > Replication was to an empty database btw. Not sure why this problem > never happened in previous versions though... Could it be because the > connection is relatively slow? (Remote server is in the US, I'm from > the Netherlands. Loading the remote database url in a browser takes > about 1 second.) > > > On Fri, Mar 20, 2009 at 12:37, Adam Kocoloski > wrote: >> On Mar 20, 2009, at 6:07 AM, Jaap van der Plas wrote: >> >>> After upgrading to the new database format (using a script that >>> simply >>> read docs from the old database, and created them as new documents >>> in >>> the new database) I am no longer able to replicate a certain >>> database >>> from a remote server to my local machine. Using an ssh tunnel I >>> access >>> the remote database on localhost:5985. >>> >>> Using the replicator interface on the local instance of couchdb, I >>> am >>> able to replicate from my local database, to the remote database. >>> However, the reverse doesn't work: the _replicate post request >>> takes a >>> very time / doesn't seem to finish at all (seen with firebug.) The >>> database in question is 1mb in size (compacted.) If I try >>> replicating >>> with a db with 1 doc in it it does work both ways. Where should I be >>> looking for the cause of this problem? Is it possible there is some >>> offending/corrupt document in the database? >>> >>> regards, >>> Jaap >> >> Hi Jaap, is there anything in the logfile for the local instance? >> The >> replicator is usually pretty chatty about any problems. Best, >> >> Adam >>