Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of virtualandy@gmail.com
 designates 209.85.214.173 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAL+Y1nuP=wBwXn8eM7MBzZg2v3nKChTEVmo=BNtwHF5ukFi+Dw@mail.gmail.com>
References: 
 <CALfshb2baKZ+tOHut_GVsaLBcbfhicgcKxapv4KxzjrkjeZYjw@mail.gmail.com>
	<CAL+Y1nuP=wBwXn8eM7MBzZg2v3nKChTEVmo=BNtwHF5ukFi+Dw@mail.gmail.com>
Date: Tue, 14 May 2013 08:00:51 -0600
Message-ID: 
 <CALfshb2A13ghpOW0+gq0dnsYPguTtz485h+qe=gFz0-O0NYsNw@mail.gmail.com>
Subject: Re: Having trouble replicating with a large couch instance (npm repo)
From: andy e <virtualandy@gmail.com>
To: user@couchdb.apache.org
Content-Type: multipart/alternative; boundary=e89a8fb1eb189bd87304dcae0f9f

--e89a8fb1eb189bd87304dcae0f9f
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Dave,

Thanks for the tips/feedback!


On Tue, May 14, 2013 at 3:33 AM, Dave Cottlehuber <dch@jsonified.com> wrote=
:

> On 14 May 2013 05:30, andy e <virtualandy@gmail.com> wrote:
>

> > Hi,
>
>
>
> Based on the awesome feedback I got from the nodejs list (see
>
> https://groups.google.com/d/msg/nodejs/sX4mbsRPwls/WtDDE-To2o4J), we trie=
d
>
> replicating with their couch instance so we could use it in an offline
>
> environment.
>
>
>
> We're essentially following the instructions at
>
> http://clock.co.uk/tech-blogs/how-to-create-a-private-npmjs-repository bu=
t
>
> replication fails after syncing about 17k documents.
>
>
>
> We've tried reinstalling couch (found one issue that suggested using a
>
> patched version of SpiderMonkey) but the same thing keeps happening, even
>
> after restarting replication several times.
>
>
>
> Here's our setup:
>
>
>
> CentOS 6.4
>
> CouchDB 1.3
>
> SpiderMonkey 1.8.5-7
>
>
>
> Replication works fine for over 17,000 documents, then we see this error
>
> and can't get past it:
>
>
>
> [Sat, 11 May 2013 00:55:39 GMT] [error] [<0.12970.4>] Replicator: couldn'=
t
>
> write document `bufferhelper`, revision
>
> `19-d339684ee7f5eaf4cc18d84da753832d`, to target database `registry`.
>
> Error: `unauthorized`, reason: `Please log in before writing to the db`.
>
>
>
> Any ideas?
>
>
>
> Thanks,
>
>
>
> Andy
>
>
>

> Hmm, weird.


> curl -svX GET http://isaacs.iriscouch.com/registry/bufferhelper |

json_reformat looks ok to me, as does:


> curl -svX GET

http://registry.npmjs.org/bufferhelper/-/bufferhelper-0.2.0.tgz

| tar tzf -

> && echo OK


> And my local npm mirror is up to date, so let's assume that the issue is =
at

your end for the moment.


> You can replicate single (or multiple) specific documents, instead of the

whole npm repo, using this, where doc_ids is an array of doc ids you want

to replicate:


> curl -svX POST http://admin:passwd@127.0.0.1:5984/_replicate -d
> '{"source":"

http://isaacs.iriscouch.com/registry/

", "target":"mirror",

> "create_target":true,  "doc_ids":["bufferhelper"]}'

-HContent-Type:application/json | json_reformat


> And if that fails, can you post the output + anything of interest in

couch.log please, along with how are you setting up your replication? e.g.

curl =85. as above.


We didn't have an admin user set up, so we added that and tried basically
this command and it seemed to go past 17k, so maybe that was the trick.


>
> Some tips:


> If you're using the replicator db (via POST _replicator endpoint), I'd

suggest ensuring you have appropriate permissions in the request, e.g.


> curl -svX PUT http://admin:passwd@127.0.0.1:5984/_replicator/mirror_npm -=
d

'{"source":"http://isaacs.iriscouch.com/registry/", "target":"mirror",

"user_ctx": {"name": "admin"}, "doc_ids":["bufferhelper"]}'

-HContent-Type:application/json


>
> You can obviously add "create_target":true,  and "continuous": true as

needed as usual.


> In your case of needing periodic updates, I'd not use continuous

replication, as the normal one will return from the API call only when

finished successfully, or failed.


> You may well find that https://npmjs.org/package/npm-gh or

http://debuggable.com/posts/private-npm-modules:4e68cc7d-1ac4-42d9-995a-343=
dcbdd56cbis

a more suitable option.


I wish we could use this, but we need to be able to install 'public'
modules (say express, which has dependencies on other stuff obviously) in
addition to our own. But, we can't access the npm repo (or github) from
where we need to do this, hence wanting a local copy. Kind of stinks but
it's life for us at the moment.


> For publishing  modules to different registries, consider doing alias

mynpm=3D'npm --userconfig /path/to/other/npmrc --registry

https://url/for/other/registry

'  from

> https://github.com/isaacs/npm/issues/2593


> A+

Dave

--e89a8fb1eb189bd87304dcae0f9f--