metron-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kwan Chong Tan <kct...@gmail.com>
Subject Re: Storm Topology Hostname Issues
Date Thu, 28 Jul 2016 08:40:31 GMT
Hi all,

I have narrowed this down to the GrokParser trying to retrieve the Grok
Pattern from HDFS but for some reason it's still referencing the invalid
ec2-54-213-184 hostname. Does anyone know where Configuration is getting
the Hadoop FS url from? All the core-default.xml and core-site.xml files
are referring to the correct internal hostname so I'm confused why the
GrokParser is still referring to the invalid hostname. Any assistance would
be much appreciated. Thanks!

Regards,
KC

public InputStream openInputStream(String streamName) throws IOException {
  FileSystem fs = FileSystem.get(new Configuration());
  Path path = new Path(streamName);
  if(fs.exists(path)) {
    return fs.open(path);
  } else {
    return getClass().getResourceAsStream(streamName);
  }
}


java.lang.RuntimeException: Grok parser Error: Call From
ip-10-0-0-21.us-west-2.compute.internal/10.0.0.21 to
ec2-54-213-184-142.us-west-2.compute.amazonaws.com:8020 failed on socket
timeout exception: org.apache.hadoop.net.ConnectTimeoutException: 20000
millis timeout while waiting for channel to be ready for connect. ch :
java.nio.channels.SocketChannel[connection-pending
remote=ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020];
For more details see: http://wiki.apache.org/hadoop/SocketTimeout at
org.apache.metron.parsers.GrokParser.init(GrokParser.java:127) at
org.apache.metron.parsers.bolt.ParserBolt.prepare(ParserBolt.java:90) at
backtype.storm.daemon.executor$fn__7143$fn__7156.invoke(executor.clj:751)
at backtype.storm.util$async_loop$fn__1985.invoke(util.clj:477) at
clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.net.ConnectTimeoutException: Call From
ip-10-0-0-21.us-west-2.compute.internal/10.0.0.21 to
ec2-54-213-184-142.us-west-2.compute.amazonaws.com:8020 failed on socket
<http://ip-10-0-0-21.us-west-2.compute.internal:8000/log?file=squid-11-1469690590-worker-6704.log>

On Wed, Jul 27, 2016 at 2:48 PM, Kwan Chong Tan <kctan6@gmail.com> wrote:

> Hi all,
>
> I have Metron deployed on a single node on AWS. Recently I tried updating
> the hostname to use AWS private DNS instead of the public DNS (which
> changes with each reboot). I think I have got most of the services working
> after updating the hostname but I have still have some issues with Storm.
>
> A sample of the storm worker logs are copied below. In particular, the
> o.a.h.i.Client seems to be still referring to the old public EC2 domain
> name but I have been unable to figure out where that config is specified.
> Could someone assist in pointing me to where that particular variable is
> stored?
>
> Thanks,
> KC
>
> 2016-07-27 06:41:21.625 s.k.ZkCoordinator [INFO] Task [1/1] Deleted
> partition managers: []
> 2016-07-27 06:41:21.625 s.k.ZkCoordinator [INFO] Task [1/1] New partition
> managers: []
> 2016-07-27 06:41:21.625 s.k.ZkCoordinator [INFO] Task [1/1] Finished
> refreshing
> 2016-07-27 06:41:22.253 b.s.m.n.Server [INFO] Getting metrics for server
> on port 6704
> 2016-07-27 06:41:24.037 o.a.h.i.Client [INFO] Retrying connect to server:
> ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020.
> Already tried 32 time(s); maxRetries=45
> 2016-07-27 06:41:44.058 o.a.h.i.Client [INFO] Retrying connect to server:
> ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020.
> Already tried 33 time(s); maxRetries=45
> 2016-07-27 06:42:04.078 o.a.h.i.Client [INFO] Retrying connect to server:
> ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020.
> Already tried 34 time(s); maxRetries=45
> 2016-07-27 06:42:21.626 s.k.ZkCoordinator [INFO] Task [1/1] Refreshing
> partition manager connections
> 2016-07-27 06:42:21.627 s.k.DynamicBrokersReader [INFO] Read partition
> info from zookeeper:
> GlobalPartitionInformation{partitionMap={0=ip-10-0-0-21.us-west-2.compute.internal:6667}}
> 2016-07-27 06:42:21.627 s.k.KafkaUtils [INFO] Task [1/1] assigned
> [Partition{host=ip-10-0-0-21.us-west-2.compute.internal:6667, partition=0}]
> 2016-07-27 06:42:21.628 s.k.ZkCoordinator [INFO] Task [1/1] Deleted
> partition managers: []
> 2016-07-27 06:42:21.628 s.k.ZkCoordinator [INFO] Task [1/1] New partition
> managers: []
> 2016-07-27 06:42:21.628 s.k.ZkCoordinator [INFO] Task [1/1] Finished
> refreshing
> 2016-07-27 06:42:22.254 b.s.m.n.Server [INFO] Getting metrics for server
> on port 6704
> 2016-07-27 06:42:24.104 o.a.h.i.Client [INFO] Retrying connect to server:
> ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020.
> Already tried 35 time(s); maxRetries=45
> 2016-07-27 06:42:44.121 o.a.h.i.Client [INFO] Retrying connect to server:
> ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020.
> Already tried 36 time(s); maxRetries=45
> 2016-07-27 06:43:04.139 o.a.h.i.Client [INFO] Retrying connect to server:
> ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020.
> Already tried 37 time(s); maxRetries=45
> 2016-07-27 06:43:21.629 s.k.ZkCoordinator [INFO] Task [1/1] Refreshing
> partition manager connections
> 2016-07-27 06:43:21.630 s.k.DynamicBrokersReader [INFO] Read partition
> info from zookeeper:
> GlobalPartitionInformation{partitionMap={0=ip-10-0-0-21.us-west-2.compute.internal:6667}}
> 2016-07-27 06:43:21.631 s.k.KafkaUtils [INFO] Task [1/1] assigned
> [Partition{host=ip-10-0-0-21.us-west-2.compute.internal:6667, partition=0}]
>
>

Mime
View raw message