flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juhani Connolly <juhani_conno...@cyberagent.co.jp>
Subject Re: Channel Exception
Date Tue, 24 Jul 2012 08:51:55 GMT
This is one possibility. It is also possible that your sink is not 
keeping up with your source. If It can't take items out faster than the 
source can put them in, events will pile up and the channel will fill up.

On 07/24/2012 04:59 PM, Stern, Mark wrote:
> You could try increasing the channel capacity. Add the following to your config file:
>
> host1.channels.memoryChannel.capacity = 500
>
> ________________________________________
> From: shekhar sharma [shekhar2581@gmail.com]
> Sent: Tuesday, July 24, 2012 9:54 AM
> To: flume-user
> Subject: Channel Exception
>
> Hello,
> I am trying to send the events to the Avro Source through external RPC client..
> While sending the events, the following error is thrown
>
> org.apache.flume.ChannelException: Space for commit to queue couldn't be acquired Sinks
are likely not keeping up with sources, or the buffer size is too tight
>   at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:85)
>          at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
>          at org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:252)
>          at org.apache.flume.source.AvroSource.append(AvroSource.java:203)
>          at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
>          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>          at java.lang.reflect.Method.invoke(Method.java:597)
>          at org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:88)
>          at org.apache.avro.ipc.Responder.respond(Responder.java:149)
>          at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:158)
>          at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
>          at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:143)
>
> I have also changed the heap size to 1024m
>
> And my host file is as follows:
>
> host1.sources = avroSource
> host1.channels = memoryChannel
> host1.sinks = esper
> #avroSource configuration
> host1.sources.avroSource.type = avro
> host1.sources.avroSource.bind = localhost
> host1.sources.avroSource.port = 41414
> host1.sources.avroSource.channels = memoryChannel
> #Channels
> host1.channels.memoryChannel.type = memory
>
> #Sinks
> host1.sinks.esper.type = com.gsshop.bigdata.poc.flume.EsperSink
> host1.sinks.esper.channel = memoryChannel
>
> What could be the reason for this error?
>
> Regards,
> SOm
>



Mime
View raw message