couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sgoto <samuelg...@gmail.com>
Subject couchdb memory issues/leaks with validators and 20MB+ json docs
Date Mon, 30 Aug 2010 06:34:52 GMT
Hey everyone,

     I'm using couchdb to store docs that are somewhat large (20MB+), but
within the configured max size.

     Storing the docs isn't a problem, couchdb seems to handle it fine. I am
having problems when using function validators and couchdb hanging my
machine after all the memory resources are consumed on PUTs.

     Below is a quick explanation of the issue I'm seeing.

     Ideas ?

     sam


how to reproduce:

1) create a db called testdb

2) create an empty javascript validator function

function(newDoc, oldDoc, user) {}

3) create a fake 20MB doc

if=/dev/zero of=test.mp3 bs=1024 count=20000
echo "{\"hello\":\"" > test.json; echo `base64 test.mp3` >> test.json;  echo
"\"}" >> test.json;

4) send it to couchdb

curl -X PUT http://127.0.0.1:5984/testdb/foobar21 -d @test.json

5) open a memory/swap monitor and couchdb's binary consume all the memory
(stopping when the swap memory ends)

kubuntu's system monitor (memory tab) ||
top ||
watch free ||

6) remove the javascript validator

7) repeat (5) and see how everything is fine

expected results:

(5) shouldn't happen. couchdb shouldn't leak memory or consume more memory
than the size of the doc (20MB).

-- 
f u cn rd ths u cn b a gd prgmr !

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message