flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Sinclair <dsincl...@chariotsolutions.com>
Subject Re: Not able to set date %d%m%Y in Sink Directory name
Date Wed, 23 Oct 2013 17:40:05 GMT
The sink is dependent on a header with the key "timestamp" being present in
the event for this to work. What source are you using?


On Wed, Oct 23, 2013 at 11:25 AM, Deepak Subhramanian <
deepak.subhramanian@gmail.com> wrote:

> Hi ,
>
> I am trying to store my logs in folders named with date for my file_roll
> and hdfssink. For some reason when I pass %d%m%Y in the sink directory it
> is not working . Any thoughts .
>
> My Flume source is a simple HTTP Handler extended from HTTPSourceHandler
>
> tier1.sinks.filesink1.type = file_roll
> tier1.sinks.filesink1.channel = c2
> tier1.sinks.filesink1.batchSize = 100000
> tier1.sinks.filesink1.rollInterval = 0
> #tier1.sinks.filesink1.rollInterval = 86400
> tier1.sinks.filesink1.sink.directory = /projects/test/logs/%d%m%Y
>
>
> 13/10/23 16:20:36 ERROR flume.SinkRunner: Unable to deliver event.
> Exception follows.
> org.apache.flume.EventDeliveryException: Failed to open file
> /projects/test/logs/%d%m%Y/1382541251307-1 while delivering event
>  at
> org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:177)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>  at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.FileNotFoundException:
> /projects/ETHAN_FLUME/logs/%d%m%Y/1382541251307-1 (No such file or
> directory)
> at java.io.FileOutputStream.open(Native Method)
>  at java.io.FileOutputStream.<init>(Unknown Source)
> at java.io.FileOutputStream.<init>(Unknown Source)
>  at
> org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:169)
>
>
> Thanks,
> Deepak Subhramanian
>

Mime
View raw message