storm-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Radim Kolar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (STORM-387) Memory leak in worker
Date Fri, 04 Jul 2014 11:55:34 GMT

    [ https://issues.apache.org/jira/browse/STORM-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052386#comment-14052386
] 

Radim Kolar commented on STORM-387:
-----------------------------------

its leaking backtype.storm.messaging.TaskMessage objects. I cant tell if they are incoming
or outgoing. It looks like bug in flow control because in my code i have:

{code}
 conf.setMaxSpoutPending(1000)
 StormSubmitter.submitTopology("Storm-throughput-test", conf, builder.createTopology())
{code}

leaking worker is running only this small bolt:

{code}
class Rtt extends StormBolt(outputFields = List("rtt")) {

  override
  def execute(t: Tuple) = {
     val now = System.currentTimeMillis()
     val rtt = now - t.getLong(0)
     t.emit(rtt)
  }
}
{code}

> Memory leak in worker
> ---------------------
>
>                 Key: STORM-387
>                 URL: https://issues.apache.org/jira/browse/STORM-387
>             Project: Apache Storm (Incubating)
>          Issue Type: Bug
>    Affects Versions: 0.9.2-incubating
>         Environment: FreeBSD 10/i386
>            Reporter: Radim Kolar
>              Labels: leak, qa
>
> There is memory leak in worker. I can reproduce it every time with following code: 
> https://github.com/hsn10/stormtest
> worker running bolt 'rtt' only leaks memory. Deploy topology and leave it about 15 minutes
running until worker is killed by supervisor due to heartbeat timeout. It timeouts because
its busy running gc all the time as you can see in jconsole.
> I was able to do memory dump, but due to its size jhat tool was not able to load it in
reasonable time (i killed it after 30 minutes)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message