couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohit Sharma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-2058) CouchDB Memory Leak - Beam.smp
Date Tue, 11 Feb 2014 23:58:21 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898523#comment-13898523
] 

Rohit Sharma commented on COUCHDB-2058:
---------------------------------------

I see Paul what you mean, here are the stats - 

Memory Usage when I rebooted the server. - 7.2G free out of 8G 
Mem:   8061508k total,   812460k used,  7249048k free,    20368k buffers
Swap:  2064376k total,        0k used,  2064376k free,   160468k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                     
                                                                                         
                                      
 2152 couchdb   20   0  463m  17m 4448 S  0.3  0.2   0:00.98 /usr/lib64/erlang/erts-5.8.5/bin/beam.smp
-Bd -K true -A 4 -- -root /usr/lib64/erlang
   
 
Memory Usage when CouchDB finished writing data -  Memory Leak - 5G of memory was consumed
and not released.
 Mem:   8061508k total,  5190152k used,  2871356k free,   116620k buffers
Swap:  2064376k total,      684k used,  2063692k free,  2551920k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                     
                                                       
13349 couchdb   20   0 1514m 911m 4492 S  1.7  20  26:21.36 /usr/lib64/erlang/erts-5.8.5/bin/beam.smp
-Bd -K true -A 4 -- -root /usr/lib64/erlang


Memory Usage when I restarted CouchDB - ~4G of memory still in use.

Mem:   8061508k total,  4272748k used,  3788760k free,   116676k buffers
Swap:  2064376k total,      684k used,  2063692k free,  2551924k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                     
                                                       
14442 couchdb   20   0  527m  17m 4368 S  0.3  0.2   0:00.90 /usr/lib64/erlang/erts-5.8.5/bin/beam.smp
-Bd -K true -A 4 -- -root /usr/lib64/erlang


> CouchDB Memory Leak - Beam.smp
> ------------------------------
>
>                 Key: COUCHDB-2058
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-2058
>             Project: CouchDB
>          Issue Type: Bug
>      Security Level: public(Regular issues) 
>          Components: Database Core
>            Reporter: Rohit Sharma
>
> Hello,
> I am experiencing performance issue with CouchDB.
> Use Case: I am working on a process that retrieves the data from RDBMS and process them
into JSON document and POST them to the CouchDB.
> I am trying to POST around half a million documents, most of them in batches (_bulk_doc)
of 10,000 and have tried with batch of 5,000, 15,000, and 20,000.
> Whole process takes around 90-100 minutes.
> During the life of the process, Memory Consumption by CouchDB keeps on growing and memory
is not released when CouchDB has finished working.
> So if the memory consumption by CouchDB was 60% at the time process finishes, memory
consumption will remain 60% and not reducing. 
> Subsequently, when the process starts running again. memory consumption is Maxed out
and CouchDB restarts itself. This restart fails the process that I am running. Looking at
the Syslogs , I see Out Of Memory Error by the CouchDB process and killing statement.
> The CouchDb process that has the issue is the "beam.smp" of Erlang.
> At this point, I have tried upgrading the memory of the server to see if this resolves
the issue, unfortunately, the issue persists. Memory Leak is there and Usage keeps on growing
until CouchDB restarts/crashed.
> I also have tried running garbage collection from Erlang command (erlang:garbage_collect().)
line but it didn't do anything.
> At this point, I am out of ideas and not sure what is going on here. Any input/suggestion
is highly appreciated!
> Env:
> Platform: Linux (Red Hat release 6.4 (Santiago))
> CouchDB: 1.3 and have tried with 1.5 as well
> RAM: Tried with 2G, 4G, and 8G
> CPU: 2 cores
> Process:/usr/lib64/erlang/erts-5.8.5/bin/beam.smp -Bd -K true -A 4 -- -root /usr/lib64/erlang



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message