storm-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Radim Kolar (JIRA)" <>
Subject [jira] [Commented] (STORM-339) Severe memory leak to OOM when ackers disabled
Date Mon, 07 Jul 2014 22:59:34 GMT


Radim Kolar commented on STORM-339:

I have 10k msg/sec throughput without ack, 100 per sec with ack, using different number of
unconfirmed messages did not change anything. This made storm unusable for me and i am exploring
different solution.

> Severe memory leak to OOM when ackers disabled
> ----------------------------------------------
>                 Key: STORM-339
>                 URL:
>             Project: Apache Storm (Incubating)
>          Issue Type: Bug
>    Affects Versions: 0.9.2-incubating
>            Reporter: Jiahong Li
> Without any ackers enabled, fast component  will continuously leak memory and causing
OOM problems when target component is slow. The OOM problem can be reproduced by running this
> with command:
> {code}
> $ storm jar storm_perf_test-1.0.0-SNAPSHOT-jar-with-dependencies.jar
--spout 1 --bolt 1 --workers 2 --testTime 600 --messageSize 6400
> {code}
> And the worker childopts with {{-Xms2g -Xmx2g -Xmn512m ...}}.
> At the same time, the executed count of target component is far behind from the emitted
count of source component.  I guess it could be that netty client is buffering too much messages
in its message_queue as target component sends back OK/Failure Response too slowly. 

This message was sent by Atlassian JIRA

View raw message