flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: ERROR hdfs.HDFSEventSink: process failed. ERROR flume.SinkRunner: Unable to deliver event. Exception follows.
Date Thu, 26 Jul 2012 13:53:23 GMT
If you want to use a bucketed paths your events need to have a "timestamp"
header which tail source does not appear to add.

Brock

On Thu, Jul 26, 2012 at 6:32 AM, khadar basha <khadarskb@gmail.com> wrote:

>
> Hi
>
> we are using flume-1.1.0-cdh4.0.1 . we have the following configurations
> in flume-conf.properties.
>
> agent1Test1.sources = seqGenSrc
> agent1Test1.channels = memoryChannel
> agent1Test1.sinks = loggerSink
>
> # For each one of the sources, the type is defined
> agent1Test1.sources.seqGenSrc.type = exec
> agent1Test1.sources.seqGenSrc.command= tail -f /tomcat/logs/catalina.out
>
>
> # The channel can be defined as follows.
> agent1Test1.sources.seqGenSrc.channels = memoryChannel
>
> # Each sink's type must be defined
> agent1Test1.sinks.loggerSink.type = hdfs
> agent1Test1.sinks.loggerSink.hdfs.path =
> hdfs://hdfshost:port/data/%Y/%m/%{host}/mylogs
>
> agent1Test1.sinks.loggerSink.hdfs.fileType = DataStream
>
> #Specify the channel the sink should use
> agent1Test1.sinks.loggerSink.channel = memoryChannel
>
> # Each channel's type is defined.
> agent1Test1.channels.memoryChannel.type = memory
>
> # Other config values specific to each type of channel(sink or source)
> # can be defined as well
> # In this case, it specifies the capacity of the memory channel
> agent1Test1.channels.memoryChannel.capacity = 1000
>
>
> I am getting the following exception. Anybody facing this issue.
>
>
> *2012-07-26 13:07:52,002 ERROR hdfs.HDFSEventSink: process failed*
> *java.lang.NumberFormatException: null*
> *        at java.lang.Long.parseLong(Long.java:375)*
> *        at java.lang.Long.valueOf(Long.java:525)*
> *        at
> org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:220)
> *
> *        at
> org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:310)
> *
> *        at
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:402)*
> *        at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> *
> *        at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)*
> *        at java.lang.Thread.run(Thread.java:662)*
> *2012-07-26 13:07:52,003 ERROR flume.SinkRunner: Unable to deliver event.
> Exception follows.*
> *org.apache.flume.EventDeliveryException:
> java.lang.NumberFormatException: null*
> *        at
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:469)*
> *        at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> *
> *        at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)*
> *        at java.lang.Thread.run(Thread.java:662)*
> *Caused by: java.lang.NumberFormatException: null*
> *        at java.lang.Long.parseLong(Long.java:375)*
> *        at java.lang.Long.valueOf(Long.java:525)*
> *        at
> org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:220)
> *
> *        at
> org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:310)
> *
> *        at
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:402)*
> *        ... 3 more*
>
> --
> Thanks,
> Khadar
>
>


-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Mime
View raw message