storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristian Øllegaard <krist...@livesystems.info>
Subject Storm hostnames and AWS
Date Tue, 08 Jul 2014 11:56:12 GMT
Hi everyone,

I’m currently setting up a storm cluster on AWS, but I have some problems with how storm
supervisor connects to the peer nodes. On AWS I’m running storm in an auto scaling group,
meaning that if one of the nodes go down, it automatically re-spawns another node with the
same image. On AWS any node can disappear at any time, therefore it’s important to run a
setup similar to this. All nodes have a hostname like ip-XX-XX-XX-XX representing their IP.
However, they are not able to connect to the others hostnames, e.g. hostname ip-10-0-0-1 will
not know about the hostname ip-10-0-0-2, although it could access 10.0.0.2. This is resulting
in errors like this on the supervisors:

2014-07-08 11:43:05 b.s.d.executor [INFO] Loaded executor tasks count-bolt:[3 3]
2014-07-08 11:43:05 b.s.d.executor [INFO] Finished loading executor count-bolt:[3 3]
2014-07-08 11:43:05 b.s.d.executor [INFO] Loading executor word-spout:[5 5]
2014-07-08 11:43:05 b.s.d.executor [INFO] Loaded executor tasks word-spout:[5 5]
2014-07-08 11:43:05 b.s.d.executor [INFO] Preparing bolt count-bolt:(3)
2014-07-08 11:43:05 b.s.d.executor [INFO] Finished loading executor word-spout:[5 5]
2014-07-08 11:43:05 b.s.d.executor [INFO] Opening spout word-spout:(5)
2014-07-08 11:43:05 b.s.d.executor [INFO] Loading executor __system:[-1 -1]
2014-07-08 11:43:05 b.s.d.executor [INFO] Loaded executor tasks __system:[-1 -1]
2014-07-08 11:43:05 b.s.d.executor [INFO] Finished loading executor __system:[-1 -1]
2014-07-08 11:43:05 b.s.u.ShellProcess [INFO] Storm multilang serializer: backtype.storm.multilang.JsonSerializer
2014-07-08 11:43:05 b.s.u.ShellProcess [INFO] Storm multilang serializer: backtype.storm.multilang.JsonSerializer
2014-07-08 11:43:05 b.s.d.executor [INFO] Preparing bolt __system:(-1)
2014-07-08 11:43:05 b.s.m.n.Client [INFO] Reconnect started for Netty-Client-ip-10-0-11-36:6703...
[4]
2014-07-08 11:43:05 b.s.m.n.StormClientErrorHandler [INFO] Connection failed Netty-Client-ip-10-0-11-36:6703
java.nio.channels.UnresolvedAddressException: null
at sun.nio.ch.Net.checkAddress(Net.java:127) ~[na:1.7.0_55]
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:644) ~[na:1.7.0_55]
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:140)
~[netty-3.2.2.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:103)
~[netty-3.2.2.Final.jar:na]
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:60)
~[netty-3.2.2.Final.jar:na]
at org.jboss.netty.channel.Channels.connect(Channels.java:541) [netty-3.2.2.Final.jar:na]
at org.jboss.netty.channel.AbstractChannel.connect(AbstractChannel.java:218) [netty-3.2.2.Final.jar:na]
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227) [netty-3.2.2.Final.jar:na]
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188) [netty-3.2.2.Final.jar:na]
at backtype.storm.messaging.netty.Client.connect(Client.java:147) [storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.messaging.netty.Client.access$000(Client.java:42) [storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.messaging.netty.Client$1.run(Client.java:104) [storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_55]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_55]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
[na:1.7.0_55]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_55]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55]


I’m a bit unsure how to proceed. Is it possible to form to register the IP instead of the
hostname somehow? Or is there another way to do this?

Kind regards,

Kristian
Mime
View raw message