Mailing-List: contact user-help@flume.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flume.apache.org
Received-SPF: pass (athena.apache.org: local policy includes SPF record at
 spf.trusted-forwarder.org)
MIME-Version: 1.0
In-Reply-To: 
 <CA+UubihRmonOncA9-56KUVsDx6tx30E8c7VF2i4TyNuaL_k4PQ@mail.gmail.com>
References: 
 <CA+UubihRmonOncA9-56KUVsDx6tx30E8c7VF2i4TyNuaL_k4PQ@mail.gmail.com>
Date: Wed, 23 Oct 2013 13:40:05 -0400
Message-ID: 
 <CAGtiFWXn+6PXy6U32N-SBW7pUN8ue3ME3ALvmdW+tMk5UU57nA@mail.gmail.com>
Subject: Re: Not able to set date %d%m%Y in Sink Directory name
From: David Sinclair <dsinclair@chariotsolutions.com>
To: user@flume.apache.org
Content-Type: multipart/alternative; boundary=089e012282c8fded3a04e96c0123

--089e012282c8fded3a04e96c0123
Content-Type: text/plain; charset=ISO-8859-1

The sink is dependent on a header with the key "timestamp" being present in
the event for this to work. What source are you using?


On Wed, Oct 23, 2013 at 11:25 AM, Deepak Subhramanian <
deepak.subhramanian@gmail.com> wrote:

> Hi ,
>
> I am trying to store my logs in folders named with date for my file_roll
> and hdfssink. For some reason when I pass %d%m%Y in the sink directory it
> is not working . Any thoughts .
>
> My Flume source is a simple HTTP Handler extended from HTTPSourceHandler
>
> tier1.sinks.filesink1.type = file_roll
> tier1.sinks.filesink1.channel = c2
> tier1.sinks.filesink1.batchSize = 100000
> tier1.sinks.filesink1.rollInterval = 0
> #tier1.sinks.filesink1.rollInterval = 86400
> tier1.sinks.filesink1.sink.directory = /projects/test/logs/%d%m%Y
>
>
> 13/10/23 16:20:36 ERROR flume.SinkRunner: Unable to deliver event.
> Exception follows.
> org.apache.flume.EventDeliveryException: Failed to open file
> /projects/test/logs/%d%m%Y/1382541251307-1 while delivering event
>  at
> org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:177)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>  at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.FileNotFoundException:
> /projects/ETHAN_FLUME/logs/%d%m%Y/1382541251307-1 (No such file or
> directory)
> at java.io.FileOutputStream.open(Native Method)
>  at java.io.FileOutputStream.<init>(Unknown Source)
> at java.io.FileOutputStream.<init>(Unknown Source)
>  at
> org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:169)
>
>
> Thanks,
> Deepak Subhramanian
>

--089e012282c8fded3a04e96c0123
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">The sink is dependent on a header with the key &quot;times=
tamp&quot; being present in the event for this to work. What source are you=
 using?</div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">=
On Wed, Oct 23, 2013 at 11:25 AM, Deepak Subhramanian <span dir=3D"ltr">&lt=
;<a href=3D"mailto:deepak.subhramanian@gmail.com" target=3D"_blank">deepak.=
subhramanian@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi ,=A0<div><br></div><div>=
I am trying to store my logs in folders named with date for my file_roll an=
d hdfssink. For some reason when I pass %d%m%Y in the sink directory it is =
not working . Any thoughts .=A0</div>

<div><br></div><div>My Flume source is a simple HTTP Handler extended from =
HTTPSourceHandler</div>


<div><br></div><div><div>tier1.sinks.filesink1.type =3D file_roll</div><div=
>tier1.sinks.filesink1.channel =3D c2</div><div>tier1.sinks.filesink1.batch=
Size =3D 100000</div><div>tier1.sinks.filesink1.rollInterval =3D 0</div><di=
v>#tier1.sinks.filesink1.rollInterval =3D 86400</div>

<div>tier1.sinks.filesink1.sink.directory =3D /projects/test/logs/%d%m%Y</d=
iv><div><br></div><div><br></div><div><div>13/10/23 16:20:36 ERROR flume.Si=
nkRunner: Unable to deliver event. Exception follows.</div><div>org.apache.=
flume.EventDeliveryException: Failed to open file /projects/test/logs/%d%m%=
Y/1382541251307-1 while delivering event</div>

<div><span style=3D"white-space:pre-wrap">	</span>at org.apache.flume.sink.=
RollingFileSink.process(RollingFileSink.java:177)</div><div><span style=3D"=
white-space:pre-wrap">	</span>at org.apache.flume.sink.DefaultSinkProcessor=
.process(DefaultSinkProcessor.java:68)</div>

<div><span style=3D"white-space:pre-wrap">	</span>at org.apache.flume.SinkR=
unner$PollingRunner.run(SinkRunner.java:147)</div><div><span style=3D"white=
-space:pre-wrap">	</span>at java.lang.Thread.run(Unknown Source)</div>
<div>Caused by: java.io.FileNotFoundException: /projects/ETHAN_FLUME/logs/%=
d%m%Y/1382541251307-1 (No such file or directory)</div><div><span style=3D"=
white-space:pre-wrap">	</span>at java.io.FileOutputStream.open(Native Metho=
d)</div>

<div><span style=3D"white-space:pre-wrap">	</span>at java.io.FileOutputStre=
am.&lt;init&gt;(Unknown Source)</div><div><span style=3D"white-space:pre-wr=
ap">	</span>at java.io.FileOutputStream.&lt;init&gt;(Unknown Source)</div>

<div><span style=3D"white-space:pre-wrap">	</span>at org.apache.flume.sink.=
RollingFileSink.process(RollingFileSink.java:169)</div></div><div><br></div=
><div><br></div>Thanks,=A0<span class=3D"HOEnZb"><font color=3D"#888888"><b=
r>Deepak Subhramanian
</font></span></div></div>
</blockquote></div><br></div>

--089e012282c8fded3a04e96c0123--