hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Wilkinson <paul.m.wilkin...@gmail.com>
Subject Re: Flume port issue
Date Tue, 21 May 2013 15:29:12 GMT
This could be a myriad of things, but some points to note:

1. Flume has its own mailing list - I've moved your post there instead.
2. Your command line suggests connecting to localhost:4000, but the log shows ubuntu:41414
instead (different attempts maybe?)
3. The correct type for file roll sink is "file_roll", not "FILE_ROLL".

I'd suggest taking a look at the logs for the agent itself, rather than the Avro Client -
chances are the agent isn't starting up due to the config problems.

Paul


On 21 May 2013, at 08:47, Sai Sai <saigraph@yahoo.in> wrote:

> Just a friendly follow up to see if anyone has any suggestions for the issue with port
given below.
> Any help is appreciated.
> Thanks
> Sai
> 
> On May 20, 2013 5:40 PM, "Sai Sai" <saigraph@yahoo.in> wrote:
> Not sure if this is the right group to ask questions about flume:
> 
> I am getting an exception about unable to open a port in flume when trying to create
a remote agent, more details below:
> ---------------------------
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to Flume. Exception
follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port: 41414 }: RPC
connection error
> ---------------------------
> 
> Here r the steps i have followed:
> 
> Step 1: Here is my agent3.conf created in the flume/conf dir:
> 
> ******
> agent3.sources = avrosource
> agent3.sinks = filesink
> agent3.channels = jdbcchannel
> 
> agent3.sources.avrosource.type = avro
> agent3.sources.avrosource.bind = localhost
> agent3.sources.avrosource.port = 4000
> agent3.sources.avrosource.threads = 5
> 
> agent3.sinks.filesink.type = FILE_ROLL
> agent3.sinks.filesink.sink.directory = /home/satish/work/apache-flume-1.3.1-bin/files
> agent3.sinks.filesink.sink.rollInterval = 0
> 
> agent3.channels.jdbcchannel.type = jdbc
> 
> agent3.sources.avrosource.channels = jdbcchannel
> agent3.sinks.filesink.channel = jdbcchannel
> 
> ******
> 
> Step 2: Then i have saved it successfully and created a new test file like this:
> 
> Step 3: echo "Hello World" > /home/satish/message3
> 
> Step 4: Tried executing this command:
> 
> ./flume-ng avro-client -H localhost -p 4000 -F /home/satish/message3
> 
> I get this exception below, please help:
> 
> ------------------------------
> 
> Djava.library.path=:/home/satish/work/hadoop-1.0.4/libexec/../lib/native/Linux-i386-32
org.apache.flume.client.avro.AvroCLIClient -H ubuntu -p 41414 -F /usr/logs/log.10
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to Flume. Exception
follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port: 41414 }: RPC
connection error
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> 
> 	at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> 	at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
> 	at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:180)
> 	at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:71)
> Caused by: java.io.IOException: Error connecting to ubuntu/127.0.0.1:41414
> 	at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
> 	at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
> 	at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> 	... 5 more
> Caused by: java.net.ConnectException: Connection refused
> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> 	at java.lang.Thread.run(Thread.java:679)
> 
> 
> Please help.
> Thanks
> Sai
> 
> 
> 
> 
> 
> 
> 
> 

Mime
View raw message