incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: Cannot allocate 729810240 bytes of memory
Date Sun, 28 Mar 2010 23:30:56 GMT
Hi Terry,

Erlang is designed to crash when malloc() fails. There is no sensible way
to recover from that, What you want to do is chopping up the 750k docs
into smaller chunks of say 1-10k and do multiple bulk inserts. You can
play with chunk size and concurrent bulk requests to find the fastest
numbers.

Cheers
Jan
--

On 28 Mar 2010, at 16:28, Terry Cullen wrote:

> Hi guys,
> 
> Using CouchDB 0.10.1 and Erlang  R13B01 (erts-5.7.2) on Ubuntu 9.1.
> 
> I am attempting to insert 750, 000 documents using the bulk_docs API at 500
> documents per post.  Couch is crashing with the following dump:
> Mon Mar 29 10:16:32 2010
> Slogan: eheap_alloc: Cannot allocate 729810240 bytes of memory (of type
> "heap").
> System version: Erlang R13B01 (erts-5.7.2) [source] [smp:4:4] [rq:4]
> [async-threads:0] [kernel-poll:true]
> Compiled: Tue Oct 20 09:38:33 2009
> Atoms: 7905
> =memory
> total: 1276194416
> processes: 839980176
> processes_used: 839973872
> system: 436214240
> atom: 342109
> atom_used: 338824
> binary: 431979480
> code: 2534675
> ets: 219780
> 
> Is there a way to tune it to not crash?
> 
> Terry


Mime
View raw message