couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Levin <>
Subject couchdb replication memory consumption
Date Sun, 17 Nov 2013 22:13:36 GMT

I recently compiled and installed CouchDB 1.5.0 on Ubuntu 12.04 with the
intention or creating my own replica of the NPM registry. I followed the
official NPM ( and kicked off
replication of the NPM database (50gb+):

curl -X POST http://@ -d '{"source":"", "target":"registry",
"continuous":true, "create_target":true}' -H "Content-Type:

Replication was proceeding smoothly for a few days until the database
reached 48gb. From this point forward replication crashes every time after
a few minutes with the following error:

[Sun, 17 Nov 2013 22:10:25 GMT] [error] [<0.609.0>] ** Generic server
<0.609.0> terminating
** Last message in was {#Port<0.3446>,{exit_status,137}}
** When Server state == {os_proc,"/usr/local/bin/couchjs
** Reason for termination ==
** {exit_status,137}

[Sun, 17 Nov 2013 22:10:29 GMT] [error] [<0.609.0>] {error_report,<0.31.0>,


Further investigation revealed that after replication is invoked, process:


starts consuming all available system memory within a few minutes (1.5gb
ram + 1gb swap) until a crash and an OOM error above.

Result from _active_tasks prior to crash:


Tried reducing worker_processes to 1 and worker_batch_size to 100 with no

At this point not sure if this behavior might be due to a memory leak,
insufficient resources, or a misconfiguration.

Any help would be appreciated,


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message