Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Date: Tue, 11 Feb 2014 21:21:19 +0000 (UTC)
From: "Paul Joseph Davis (JIRA)" <jira@apache.org>
To: dev@couchdb.apache.org
Message-ID: <JIRA.12694586.1392151019348.26439.1392153679650@arcas>
In-Reply-To: <JIRA.12694586.1392151019348@arcas>
References: <JIRA.12694586.1392151019348@arcas>
Subject: [jira] [Commented] (COUCHDB-2058) CouchDB Memory Leak - Beam.smp
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/COUCHDB-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898325#comment-13898325 ] 

Paul Joseph Davis commented on COUCHDB-2058:
--------------------------------------------

Rohit, can you do that process before and after you see CouchDB collecting large amounts of RAM? Unless you specifically looked at top before running it would appear to have been a mostly idle instance that didn't have much garbage. If you did look at top and it had gigabytes allocated then we may have a real leak in a NIF somewhere.

I'd be a bit surprised that a NIF is leaking given we haven't changed them for a few releases to my knowledge.

> CouchDB Memory Leak - Beam.smp
> ------------------------------
>
>                 Key: COUCHDB-2058
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-2058
>             Project: CouchDB
>          Issue Type: Bug
>      Security Level: public(Regular issues) 
>          Components: Database Core
>            Reporter: Rohit Sharma
>
> Hello,
> I am experiencing performance issue with CouchDB.
> Use Case: I am working on a process that retrieves the data from RDBMS and process them into JSON document and POST them to the CouchDB.
> I am trying to POST around half a million documents, most of them in batches (_bulk_doc) of 10,000 and have tried with batch of 5,000, 15,000, and 20,000.
> Whole process takes around 90-100 minutes.
> During the life of the process, Memory Consumption by CouchDB keeps on growing and memory is not released when CouchDB has finished working.
> So if the memory consumption by CouchDB was 60% at the time process finishes, memory consumption will remain 60% and not reducing. 
> Subsequently, when the process starts running again. memory consumption is Maxed out and CouchDB restarts itself. This restart fails the process that I am running. Looking at the Syslogs , I see Out Of Memory Error by the CouchDB process and killing statement.
> The CouchDb process that has the issue is the "beam.smp" of Erlang.
> At this point, I have tried upgrading the memory of the server to see if this resolves the issue, unfortunately, the issue persists. Memory Leak is there and Usage keeps on growing until CouchDB restarts/crashed.
> I also have tried running garbage collection from Erlang command (erlang:garbage_collect().) line but it didn't do anything.
> At this point, I am out of ideas and not sure what is going on here. Any input/suggestion is highly appreciated!
> Env:
> Platform: Linux (Red Hat release 6.4 (Santiago))
> CouchDB: 1.3 and have tried with 1.5 as well
> RAM: Tried with 2G, 4G, and 8G
> CPU: 2 cores
> Process:/usr/lib64/erlang/erts-5.8.5/bin/beam.smp -Bd -K true -A 4 -- -root /usr/lib64/erlang


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)