couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Evans" <j...@jpevans.com>
Subject Erlang crash on memory allocation attempt
Date Fri, 04 Jul 2008 05:37:06 GMT
Hi,

I'm really eager to write up a great couchdb success story about how my
stealth mode startup has used couchdb to great effect.... but I can't yet,
since there is at least one show stopper for me -- specifically, this:

[info] [<0.84.0>] 127.0.0.1 - - "GET /products/_design%2Fproduct_with_offer"
304

Crash dump was written to: erl_crash.dump
eheap_alloc: Cannot allocate 2280657000 bytes of memory (of type "heap").
Aborted

Which I get when it tries to update my view, which looks like this
(cut/pasted from Futon):

{
   "all": {
       "map": "\n\t\t\t\tfunction(doc) { \n\t\t\t\t\tif ('Product' ==
doc.type) {\n\t\t\t\t\t\temit([doc._id, 0], doc)\n\t\t\t\t\t} else if
('Offer' == doc.type) {\n\t\t\t\t\t\temit([doc.product, 1],
doc)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t"
   }
}

Here it is again from the source code I use to create it (which uses my home
grown client library but should be easy to understand and is more readable):

    create_or_replace_view(database, "product_with_offer", {
        "all": {
            "map": """
                function(doc) {
                    if ('Product' == doc.type) {
                        emit([doc._id, 0], doc)
                    } else if ('Offer' == doc.type) {
                        emit([doc.product, 1], doc)
                    }
                }
            """
        }
    })

The database has 12,492 documents and is compacted to a size of 85.4m.

The machine this is running on only has 2 gigs of memory and 1 gig of swap
with over 1gig of other stuff running all the time, so obviously the basic
problem is that couchdb is trying to allocate more memory than is available
and one solution would be to add RAM to the box, but it really seems to me
as if generating a view on a dataset that is 85.4 megs should not require
over 2 gigs of RAM.  Am I doing something stupid in my view code?  I am out
of ideas on what to investigate next.  Any ideas?

Thanks,
-
John

P.S. There are a few other non show-stopper issues that I've run into with
couchdb (e.g. the document counts in Futon being wrong in various
situations) but other than the show stopper above, it looks as if it
would/will be of great value to our product development effort.... so much
so in fact, that thanks to some ugly workarounds (untenable in the long run,
but doing the trick in the short run) even with this "show stopper" we are
already using it very successfully to accelerate our development... and we
have just barely scratched the surface of it's capabilities I am sure.  So
you can consider this a minor success story already. :)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message