couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mat Degerholm <Mat.Degerh...@kiwirail.co.nz>
Subject RE: Assistance in tuning CouchDB heap and/or replicator config
Date Wed, 29 Oct 2014 22:37:28 GMT
Hi again,

Apologies for the re-send but as I didn't receive any replies to my earlier message I wanted
to check it hadn't slipped through the cracks.  Can anyone advise of a way to increase the
Couch/Erlang heap?

Regards,
Mat


From: Mat Degerholm
Sent: Wednesday, 22 October 2014 5:25 p.m.
To: 'user@couchdb.apache.org'
Cc: Mat Degerholm
Subject: Assistance in tuning CouchDB heap and/or replicator config

Hi guys,

Just looking for a little assistance in tuning the CouchDB heap and/or replicator config.

When performing an initial/full replication to an empty 2nd node using the _replicator database,
the 2nd node crashes with the following error:

hend=0x00007f280d67f628
stop=0x00007f280d67f418
htop=0x00007f280d6a27b0
heap=0x00007f280d592028
beam/erl_gc.c, line 427: <0.194.0>: Overrun stack and heap
heart: Tue Oct 21 13:29:55 2014: Erlang has closed.
heart: Tue Oct 21 13:29:56 2014: Executed "/opt/couchdb/bin/couchdb -k" -> 0. Terminating.

As my replication source node does have a sizeable data set (16,000 docs, 180,000 updates,
total size around 1GB), I found I could prevent the crash by reducing the replication worker
batch size.  But I found I had to drop down 2 orders of magnitude (from the default of 500
down to 5) before the problem disappeared!

[replicator]
; With lower batch sizes checkpoints are done more frequently. Lower batch sizes
; also reduce the total amount of used RAM memory.
worker_batch_size = 5

I'm nervous having to tweak the replicator setup so far from the defaults, so is there any
way I can tune the stack/heap?  I have included my ulimit output in case you could recommend
any changes (note that my stack size and max memory are unlimited).

Thanks in advance,
Mat


Technical Details

CouchDB version:            1.6.0
Erlang:                                  OTP 17.0
OS:                                         RHEL 6.5
ERL_MAX_PORTS:           4096
ERL_FLAGS:                        "+A 4"


ulimit output:
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 30502
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 4096
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message