Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of mikeal.rogers@gmail.com
 designates 209.85.210.180 as permitted sender)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1250)
Subject: Re: CouchDB Crash report db_not_found when attempting to replicate
 databases
From: Mikeal Rogers <mikeal.rogers@gmail.com>
In-Reply-To: 
 <CAAL6JQhN4iS3VWots6ffuN5eveckn_91BB2KF-NY3ZLvPzrJWg@mail.gmail.com>
Date: Wed, 14 Sep 2011 13:10:43 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <EFC4BC5B-3B70-412A-934C-30974A4CB651@gmail.com>
References: 
 <CALKFbxuugLJJY-NH46U0u584L+XDqM3NGSpeNxsJyrxosPEuCg@mail.gmail.com>
 <CALKFbxvtkaPw9Czub=qgoMcUUuP=x-q60wnpcV2RUWQx7p3Ecw@mail.gmail.com>
 <CAONf+LQw=wM=R39mXLQbktBYyRQ5MBuVVfuFgg7bBYmRP-0smw@mail.gmail.com>
 <C9FCD0B9-F3E9-4281-96B4-6F78C86B8DF2@gmail.com>
 <050C91F2664C44CFA66A262EE4124ADC@gmail.com>
 <C6CFA76E-922C-4848-9CD6-0F63E253DA90@gmail.com>
 <CDE325E30E4F45EAB9A2B25DFA15C785@gmail.com>
 <CAAL6JQhN4iS3VWots6ffuN5eveckn_91BB2KF-NY3ZLvPzrJWg@mail.gmail.com>
To: user@couchdb.apache.org

HAHA! I already forgot that we did this.

-Mikeal

On Sep 14, 2011, at September 14, 201112:51 PM, Randall Leeds wrote:

> On Wed, Sep 14, 2011 at 12:19, Adam Kocoloski =
<adam.kocoloski@gmail.com>wrote:
>=20
>> There's a multipart API which allows for a single PUT request =
containing
>> the document body as JSON and all its attachments in their raw form.
>> Documentation is pretty thin at the moment, and unfortunately I think =
it
>> doesn't quite allow for a pipe(). Would be really nice if it did, =
though.
>>=20
>=20
> It does. We figured it out together a couple weeks ago and that's when =
this
> code came into being.
> Requesting a _specific_ revision with ?revs=3Dtrue will give you a
> multipart/related response suitable for passing straight into a
> ?new_edits=3Dfalse&rev=3D PUT.
> See https://github.com/mikeal/replicate/blob/master/main.js#L49
>=20
>=20
>>=20
>> On Wednesday, September 14, 2011 at 1:16 PM, Mikeal Rogers wrote:
>>=20
>>> npm is mostly attachments and I haven't seen any issues so far.
>>>=20
>>> I wish there was a better way to replicate attachments atomically =
for a
>> single revision but if there is, I don't know about it.
>>>=20
>>> It's probably a huge JSON operation and it sucks, but I don't have =
to
>> parse it in node.js, I just pipe() the body right along.
>>>=20
>>> -Mikeal
>>>=20
>>> On Sep 14, 2011, at September 14, 20118:42 AM, Adam Kocoloski wrote:
>>>=20
>>>> Hi Mikeal, I just took a quick peek at your code. It looks like you
>> handle attachments by inlining all of them into the JSON =
representation of
>> the document. Does that ever cause problems when dealing with the =
~100 MB
>> attachments in the npm repo?
>>>>=20
>>>> I've certainly seen my fair share of problems with attachment
>> replication in CouchDB 1.0.x. I have a sneaking suspicion that there =
are
>> latent bugs related to incorrect determinations of Content-Length =
under
>> various compression scenarios.
>>>>=20
>>>> Adam
>>>>=20
>>>> On Tuesday, September 13, 2011 at 5:08 PM, Mikeal Rogers wrote:
>>>>=20
>>>>> My replicator is fairly young so I think calling it "reliable" =
might
>> be a little misleading.
>>>>>=20
>>>>> It does less, I don't ever attempt to cache the high watermark =
(last
>> seq written) and start over from there. If the process crashes just =
start
>> over from scratch. This can lead to a delay after restart but I find =
that
>> it's much simpler and more reliable on failure.
>>>>>=20
>>>>> It's also simpler because it doesn't have to content with being an
>> http client and a client of the internal couchdb erlang API. It just =
proxies
>> requests from one couch to another.
>>>>>=20
>>>>> While I'm sure there are bugs that I haven't found yet in it, I =
can
>> say that it replicates the npm repository quite well and I'm using it =
in
>> production.
>>>>>=20
>>>>> -Mikeal
>>>>>=20
>>>>> On Sep 13, 2011, at September 13, 201111:44 AM, Max Ogden wrote:
>>>>>=20
>>>>>> Hi Chris,
>>>>>>=20
>>>>>> =46rom what I understand the current state of the replicator (as =
of
>> 1.1) is
>>>>>> that for certain types of collections of documents it can be
>> somewhat
>>>>>> fragile. In the case of the node.js package repository,
>> http://npmjs.org,
>>>>>> there are many relatively large (~100MB) documents that would
>> sometimes
>>>>>> throw errors or timeout during replication and crash the
>> replicator, at
>>>>>> which point the replicator would restart and attempt to pick up
>> where it
>>>>>> left off. I am not an expert in the internals of the replicator =
but
>>>>>> apparently the cumulative time required for the replicator to
>> repeatedly
>>>>>> crash and then subsequently relocate itself in _changes feed in =
the
>> case of
>>>>>> replicating the node package manager was making the built in =
couch
>>>>>> replicator unusable for the task.
>>>>>>=20
>>>>>> Two solutions exist that I know of. There is a new replicator in
>> trunk (not
>>>>>> to be confused with the _replicator db from 1.1 -- it is still
>> using the old
>>>>>> replicator algorithms) and there is also a more reliable =
replicator
>> written
>>>>>> in node.js https://github.com/mikeal/replicate that was was
>> written
>>>>>> specifically to replicate the node package repository between
>> hosting
>>>>>> providers.
>>>>>>=20
>>>>>> Additionally it may be useful if you could describe the
>> 'fingerprint' of
>>>>>> your documents a bit. How many documents are in the failing
>> databases? are
>>>>>> the documents large or small? do they have many attachments? how
>> large is
>>>>>> your _changes feed?
>>>>>>=20
>>>>>> Cheers,
>>>>>>=20
>>>>>> Max
>>>>>>=20
>>>>>> On Tue, Sep 13, 2011 at 11:22 AM, Chris Stockton
>>>>>> <chrisstocktonaz@gmail.com (mailto:chrisstocktonaz@gmail.com
>> )>wrote:
>>>>>>=20
>>>>>>> Hello,
>>>>>>>=20
>>>>>>> We now have about 150 dbs that are refusing to replicate with
>> random
>>>>>>> crashes, which provide really zero debug information. The error
>> is db
>>>>>>> not found, but I know its available. Does anyone know how can I
>>>>>>> trouble shoot this? Do we just have to many databases =
replicating
>> for
>>>>>>> couchdb to handle? 4000 is a small number for the massive
>> hardware
>>>>>>> these are running on.
>>>>>>>=20
>>>>>>> -Chris
>>=20
>>=20
>>=20