couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Cohen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-1258) eheap_alloc OOM errors when attempting to build selected views
Date Tue, 23 Aug 2011 18:14:29 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-1258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089632#comment-13089632
] 

James Cohen commented on COUCHDB-1258:
--------------------------------------

I have just restored the same database to a completely different platform/server. This server
is running Ubuntu 11.04, erlang R13B3 and CouchDB 1.0.1

I can get it to exhibit exactly the same behaviour which surprises me.

One characteristic of the database which may be unusual is the high number of deleted docs
in the database. The use case for this is that documents only have a limited life-span after
which we delete them from the system.

As you can see from the doc stats only a small number of the original docs are still there:
"doc_count":593274,"doc_del_count":4743352,"update_seq":10559287

I wonder if someone could perform a test using a similar database (one that has a high number
of deletions)

> eheap_alloc OOM errors when attempting to build selected views
> --------------------------------------------------------------
>
>                 Key: COUCHDB-1258
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1258
>             Project: CouchDB
>          Issue Type: Bug
>         Environment: CentOS 5.6, 1GB RAM (approx 800MB free)
> CouchDb 1.1.0, Erlang R14B-03, js lib 1.7.0-8 EPEL RPM build
> Couch database exhibiting this behaviour:
> {"db_name":"activity_new","doc_count":593274,"doc_del_count":4743352,"update_seq":10559287,"purge_seq":0,"compact_running":false,"disk_size":3366396013,"instance_start_time":"1314097423985726","disk_format_version":5,"committed_update_seq":10559287}
>            Reporter: James Cohen
>            Priority: Critical
>
> We spotted OOM errors crashing our CouchDb instance when attempting to rebuild selected
views. CouchDb was dying with the following messages (worth noting that the type reported
varies between heap/old_heap
> eheap_alloc: Cannot allocate 478288480 bytes of memory (of type "heap").
> eheap_alloc: Cannot allocate 597860600 bytes of memory (of type "heap").
> eheap_alloc: Cannot allocate 747325720 bytes of memory (of type "old_heap").
> eheap_alloc: Cannot allocate 597860600 bytes of memory (of type "old_heap").
> By modifying the view I was able to find a view that could consistently crash the server
and another that ran fine. They are as follows:
> Runs out of memory v.quickly
> {
>    "_id": "_design/cleanup",
>    "_rev": "5-e004fbab278355e9d08763877e5a8295",
>    "views": {
>        "byDate": {
>            "map": "function(doc) { if (! doc.action) emit([doc.date], doc); }"
>        }
>    }
> }
> Runs fine with minimal memory usage (returns 88128 docs in the view)
> {
>    "_id": "_design/cleanup",
>    "_rev": "6-3823be6b72ca2441e235addfece6900c",
>    "views": {
>        "byDate": {
>            "map": "function(doc) { if (doc.action) emit([doc.date], doc); }"
>        }
>    }
> }
> The only difference between the two is the negation of the if conditional.
> memory usage was monitored with top on the machine while the view was being built. Under
correct behaviour I could see beam.smp using just 3 or 4% of the server's memory. With the
view that causes problems that memory usage increased until the RAM/swap on the server was
exhausted (as you can see from the error messages around 500/700MB)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message