couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Smith <>
Subject Re: NPM, CouchDB and big attachments
Date Wed, 27 Nov 2013 14:07:22 GMT
I am on-record as supportive of npm's needs here but still skeptical or at
least unclear on the fix.

Pain points:

1. Replicating the registry can have problems. Best-case scenario you have
to download 100GB of data (and growing exponentially)

2. Bandwidth costs of the primary service

Both problems are aggravated by the need for legacy support. CouchDB 1.5
has replication fixes but it is still new. And the npm clients will want to
fetch attachments from the couch server even after a newer CDN-using one
comes out.

I am conflicted. I agree with the point that you shouldn't have to
replicate 100GB (and growing) data just to have a local npm web service.
But I don't want to modify replication or the HTTP lightly.

On Wed, Nov 27, 2013 at 2:14 PM, Alexander Shorin <> wrote:

> > Move attachments out of CouchDB: Work has begun to move the package
> tarballs out of
> > CouchDB and into Joyent's Manta service. Additionally, MaxCDN has
> generously offered to
> > provide CDN services for npm, once the tarballs are moved out of the
> registry database.
> > This will help improve delivery speed, while dramatically reducing the
> file system I/O load on
> > the CouchDB servers. Work is progressing slowly, because at each stage
> in the plan, we are
> > making sure that current replication users are minimally impacted.
> I wonder is it CouchDB non-optimal I/O and/or can 769 issue fix it?
> There is alpha-patch attached. May be it's good time to push it
> forward? What things are left for it?
> --
> ,,,^..^,,,

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message