couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Newson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-1258) eheap_alloc OOM errors when attempting to build selected views
Date Wed, 24 Aug 2011 20:58:31 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-1258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090478#comment-13090478
] 

Robert Newson commented on COUCHDB-1258:
----------------------------------------

you could try this to see if it helps with memory usage. Verified locally that it radically
reduces the number of checkpoints.

diff --git a/src/couchdb/couch_view_group.erl b/src/couchdb/couch_view_group.erl
index cde949b..7002d07 100644
--- a/src/couchdb/couch_view_group.erl
+++ b/src/couchdb/couch_view_group.erl
@@ -243,7 +243,7 @@ handle_cast({partial_update, Pid, NewGroup}, #group_state{updater_pid=Pid}
         group = Group
     } = State,
     NewSeq = NewGroup#group.current_seq,
-    case NewSeq > Group#group.current_seq of
+    case NewSeq > 1000 + Group#group.current_seq of
     true ->
         ?LOG_INFO("checkpointing view update at seq ~p for ~s ~s", [NewSeq,
             DbName, NewGroup#group.name]),

> eheap_alloc OOM errors when attempting to build selected views
> --------------------------------------------------------------
>
>                 Key: COUCHDB-1258
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1258
>             Project: CouchDB
>          Issue Type: Bug
>    Affects Versions: 1.1
>         Environment: CentOS 5.6, 1GB RAM (approx 800MB free)
> CouchDb 1.1.0, Erlang R14B-03, js lib 1.7.0-8 EPEL RPM build
> Couch database exhibiting this behaviour:
> {"db_name":"activity_new","doc_count":593274,"doc_del_count":4743352,"update_seq":10559287,"purge_seq":0,"compact_running":false,"disk_size":3366396013,"instance_start_time":"1314097423985726","disk_format_version":5,"committed_update_seq":10559287}
>            Reporter: James Cohen
>            Priority: Critical
>
> We spotted OOM errors crashing our CouchDb instance when attempting to rebuild selected
views. CouchDb was dying with the following messages (worth noting that the type reported
varies between heap/old_heap
> eheap_alloc: Cannot allocate 478288480 bytes of memory (of type "heap").
> eheap_alloc: Cannot allocate 597860600 bytes of memory (of type "heap").
> eheap_alloc: Cannot allocate 747325720 bytes of memory (of type "old_heap").
> eheap_alloc: Cannot allocate 597860600 bytes of memory (of type "old_heap").
> By modifying the view I was able to find a view that could consistently crash the server
and another that ran fine. They are as follows:
> Runs out of memory v.quickly
> {
>    "_id": "_design/cleanup",
>    "_rev": "5-e004fbab278355e9d08763877e5a8295",
>    "views": {
>        "byDate": {
>            "map": "function(doc) { if (! doc.action) emit([doc.date], doc); }"
>        }
>    }
> }
> Runs fine with minimal memory usage (returns 88128 docs in the view)
> {
>    "_id": "_design/cleanup",
>    "_rev": "6-3823be6b72ca2441e235addfece6900c",
>    "views": {
>        "byDate": {
>            "map": "function(doc) { if (doc.action) emit([doc.date], doc); }"
>        }
>    }
> }
> The only difference between the two is the negation of the if conditional.
> memory usage was monitored with top on the machine while the view was being built. Under
correct behaviour I could see beam.smp using just 3 or 4% of the server's memory. With the
view that causes problems that memory usage increased until the RAM/swap on the server was
exhausted (as you can see from the error messages around 500/700MB)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message