flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shekhar sharma <shekhar2...@gmail.com>
Subject Re: Unable to transport the event to Avro Source
Date Wed, 20 Jun 2012 18:00:24 GMT
Thanks hari for prompt reply. I dont have a sink that writes to avro source.
Actually i am writing a external program which will send the events to the
Avro Source. you mean to say, my external program should convert the events
into avro format and then i can send those event through a socket.

Regards,
SOm

On Wed, Jun 20, 2012 at 11:22 PM, Hari Shreedharan <
hshreedharan@cloudera.com> wrote:

>  Hi Som,
>
> AvroSource is designed to received only Avro formatted events, so if you
> write arbitrary data to it, the AvroSource will behave in an undefined way.
> If you have a sink that writes to the AvroSource, it should be using the
> Flume SDK to send the event in the Flume Avro format. If you want to write
> arbitrary data, I suggest taking a look at the netcat source.
>
> Thanks
> Hari
>
> --
> Hari Shreedharan
>
> On Wednesday, June 20, 2012 at 10:48 AM, shekhar sharma wrote:
>
> Hello,
> I am encountering an exception when i am trying to transport the events to
> flume avro source.
> Basically i have created an agent consisting of avro source and my custom
> sink.
> Previously i was using exec source, but due to its limitations i am not
> using it anymore.
>
> Since Avro Source is listening on a port, so i have created a simple
> application, which opens a socket to that port and send the events through
> this socket.
> Socket connection is successful, but when i send the events i got the
> following error:
>
> 2012-06-20 10:34:15,767 INFO source.AvroSource: Avro source
> starting:AvroSource: { bindAddress:localhost port:41414 }
> 2012-06-20 10:35:52,588 INFO ipc.NettyServer: [id: 0x00f0b4a3, /
> 127.0.0.1:46240 => /127.0.0.1:41414] OPEN
> 2012-06-20 10:35:52,612 INFO ipc.NettyServer: [id: 0x00f0b4a3, /
> 127.0.0.1:46240 => /127.0.0.1:41414] BOUND: /127.0.0.1:41414
> 2012-06-20 10:35:52,612 INFO ipc.NettyServer: [id: 0x00f0b4a3, /
> 127.0.0.1:46240 => /127.0.0.1:41414] CONNECTED: /127.0.0.1:46240
> 2012-06-20 10:35:52,792 WARN ipc.NettyServer: Unexpected exception from
> downstream.
> java.lang.OutOfMemoryError: Java heap space
>         at java.util.ArrayList.<init>(ArrayList.java:112)
>         at
> org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:154)
>         at
> org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:131)
>         at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:282)
>         at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
>         at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
>         at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
>         at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
>         at
> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
>         at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
>
> Is there something else i need to perform?
>
> Regards,
> Som Shekhar Sharma
>
>
>

Mime
View raw message