flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Johny Rufus <jru...@cloudera.com>
Subject Re: Flafka: Kafka channel unable to deliver event
Date Sat, 04 Jul 2015 17:22:14 GMT
Been looking into this for some time and found a couple of issues. Raised
Jiras for both, there are workarounds to get past these errors.

https://issues.apache.org/jira/browse/FLUME-2734
https://issues.apache.org/jira/browse/FLUME-2735

For timeout to take effect, you need to specify as -
agent1.channels.channel1.timeout = 1000000 (This is a temporary work
around, kafka.consumer.timeout.ms should work as per the guide when
FLUME-2734 is resolved)

To get past the IllegalStateException, I had to download the zookeeper jar
from http://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper/3.3.6
and put it in the lib directory. This is also a workaround to get past the
issue, until we figure out the root cause.


Thanks,
Rufus

On Fri, Jul 3, 2015 at 10:09 PM, Jun Ma <mj.saber1990@gmail.com> wrote:

> Oops, thanks for figuring that out.
> Any idea why unable to deliver event?
>
> On Fri, Jul 3, 2015 at 8:00 PM, Johny Rufus <jrufus@cloudera.com> wrote:
>
>> There is a typo in your property "cnannel1", hence the property is not
>> set
>>
>> a1.channels.cnannel1.kafka.consumer.timeout.ms = 1000000
>>
>> Thanks,
>> Rufus
>>
>> On Fri, Jul 3, 2015 at 4:49 PM, Jun Ma <mj.saber1990@gmail.com> wrote:
>>
>>> Thanks for your reply. But from what I read, the magic things Flafka
>>> does is that you don't need to have a source, you can directly move things
>>> from channel to sink.
>>>
>>> http://blog.cloudera.com/blog/2014/11/flafka-apache-flume-meets-apache-kafka-for-event-processing/
>>> (see Flume's kafka channel)
>>>
>>>
>>> On Fri, Jul 3, 2015 at 1:26 PM, Foo Lim <foo.lim@vungle.com> wrote:
>>>
>>>> There's an error that you didn't specify a source:
>>>>
>>>> Agent configuration for 'a1' has no sources.
>>>>
>>>
>>>
>>
>

Mime
View raw message