storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Sebastien Vachon <jean-sebastien.vac...@wantedanalytics.com>
Subject Migration from 0.8.3-wip to 0.9.0.1 + Netty failure
Date Wed, 08 Jan 2014 13:44:32 GMT
Hi All,

Yesterday I tried migrating from 0.8.3-wip to the latest release 0.9.0.1 + Netty and it fails.
The topology just hangs for no apparent reason ( nothing in the worker`s or supervisor`s logs.
I guess something will eventually timeout but I let it ran for about 30 minutes and no batch
have completed. Each batch usually takes about 1-2 minutes to process but not 30 minutes.

I tried the same topology under ZeroMQ works fine and I got slightly better performances over
0.8.3.

The config I used for the transport is:

storm.messaging.transport: "backtype.storm.messaging.netty.Context"
storm.messaging.netty.server_worker_threads: 1
storm.messaging.netty.client_worker_threads: 1
storm.messaging.netty.buffer_size: 5242880
storm.messaging.netty.max_retries: 31
storm.messaging.netty.max_wait_ms: 10000
storm.messaging.netty.min_wait_ms: 1000

This is basically the config I found on the net. I`ve only changed the max_retries after I
got some timeout during the startup phase.
Any idea what could cause this? Am I the only one to run into this issue? We are still using
Java 1.6 here... we had a bad experience with Java 7 and Storm a couple of months ago so I
am not eager to move this project to Java 7 but will do it if helps.

Thanks for your time and input.

Mime
View raw message