flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: Can we use escape sequences in file_roll sink?
Date Fri, 14 Jun 2013 19:34:01 GMT
The main implementation is in the BucketPath class. It should not be too difficult to create
the path based on that. 


Cheers,
Hari


On Friday, June 14, 2013 at 12:22 PM, Lenin Raj wrote:

> Hi Hari,
> 
> Here is the ticket - https://issues.apache.org/jira/browse/FLUME-1902
> 
> I will see how it is done in HDFS sink.
> 
> 
> Thanks,
> Lenin
> 
> 
> On Sat, Jun 15, 2013 at 12:40 AM, Hari Shreedharan <hshreedharan@cloudera.com (mailto:hshreedharan@cloudera.com)>
wrote:
> > Currently escaping is  not supported by File Roll Sink, but I think there is a jira
to enable it - perhaps you want to submit a patch? 
> > 
> > 
> > Cheers,
> > Hari
> > 
> > 
> > On Friday, June 14, 2013 at 12:04 PM, Lenin Raj wrote:
> > 
> > > Hello,
> > > 
> > > I tried to use %Y in my sink.directory in the file_roll sink but it resulted
in exception:
> > > 
> > > 15 Jun 2013 00:26:20,596 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.SinkRunner$PollingRunner.run:160)  - Unable to deliver event. Exception
follows.
> > > org.apache.flume.EventDeliveryException: Failed to open file /var/emaillenin/card/data/%Y/1371236170080-1
while delivering event
> > >         at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:177)
> > >         at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> > >         at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > Caused by: java.io.FileNotFoundException: /var/emaillenin/card/data/%Y/1371236170080-1
(No such file or directory)
> > >         at java.io.FileOutputStream.open(Native Method)
> > >         at java.io.FileOutputStream.<init>(FileOutputStream.java:194)
> > >         at java.io.FileOutputStream.<init>(FileOutputStream.java:145)
> > >         at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:169)
> > >         ... 3 more
> > > 
> > > When I remove %Y from the path and tried again, the files get created successfuly.
Are there any ways to use these escape sequences in file_roll sink?
> > > 
> > > Thanks,
> > > Lenin
> > 
> 


Mime
View raw message