flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roshan Naik <ros...@hortonworks.com>
Subject Re: How to configure Regex in file names flume 1.3
Date Fri, 26 Jul 2013 00:36:19 GMT
you will need to build a custom version...

1) checkout flume-1.3  sources..

2) apply this patch..

https://issues.apache.org/jira/secure/attachment/12572794/1661.patch.v9

3) build.

-roshan



On Tue, Jul 23, 2013 at 11:54 PM, Kumar, Deepak8 <deepak8.kumar@citi.com>wrote:

>  Hi Roshan,****
>
> I am using flume 1.3. Don’t we have a way to support regex in 1.3?****
>
> ** **
>
> Regards,****
>
> Deepak****
>
> ** **
>
> ** **
>
> ** **
>
> *From:* Roshan Naik [mailto:roshan@hortonworks.com]
> *Sent:* Tuesday, July 23, 2013 3:10 PM
> *To:* user@flume.apache.org
> *Cc:* Kumar, Deepak8 [CCC-OT_IT NE]; israel@aicer.org;
> cdh-user@cloudera.org
>
> *Subject:* Re: How to configure Regex in file names flume 1.3****
>
>  ** **
>
> Deepak,****
>
>    which version of flume are u using ? regex support is in 1.4, but you
> will need to use the 'shell' configuration in Exec Source for that.. see
> the User Guide for an example****
>
> ** **
>
> -roshan****
>
> ** **
>
> On Tue, Jul 23, 2013 at 10:50 AM, Kumar, Deepak8 <deepak8.kumar@citi.com>
> wrote:****
>
> Could anyone assist on this?****
>
>  ****
>
> Could we have a regex in tail –F****
>
>  ****
>
> Regards,****
>
> Deepak****
>
>  ****
>
> *From:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Sent:* Sunday, July 21, 2013 12:29 PM
> *To:* Kumar, Deepak8 [CCC-OT_IT NE]; 'user@flume.apache.org'; '
> israel@aicer.org'****
>
>
> *Subject:* RE: How to configure Regex in file names flume 1.3****
>
>  ****
>
> A simple tail works if I give the exact file name. But if I give a regex
> as file name like****
>
> agent.sources.source1.command = tail -F /var/log/flume-ng/template.*****
>
>  ****
>
> then it is not working.****
>
>  ****
>
> Regards,****
>
> Deepak****
>
>  ****
>
> *From:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Sent:* Sunday, July 21, 2013 12:27 PM
> *To:* 'user@flume.apache.org'; 'israel@aicer.org'; Kumar, Deepak8
> [CCC-OT_IT NE]
> *Subject:* RE: How to configure Regex in file names flume 1.3****
>
>  ****
>
> Hi,****
>
> I have checked it but it is not working. I verified from JConsole but it
> is not sending the log events.****
>
>  ****
>
> Do we have any other way to cross check?****
>
>  ****
>
> Regards,****
>
> Deepak****
>
>  ****
>
> *From:* Israel Ekpo [mailto:israel@aicer.org <israel@aicer.org>]
> *Sent:* Saturday, July 20, 2013 4:46 PM
> *To:* user@flume.apache.org
> *Subject:* Re: How to configure Regex in file names flume 1.3****
>
>  ****
>
> Deepak,****
>
>  ****
>
> The sample configuration you have looks valid****
>
>  ****
>
> Try it out and let us know if you are experiencing any problems****
>
>  ****
>
> More information on how to configure the Exec Source is available here.***
> *
>
>  ****
>
> http://flume.apache.org/FlumeUserGuide.html#exec-source****
>
>  ****
>
>
> ****
>
> *Author and Instructor for the Upcoming Book and Lecture Series*****
>
> *Massive Log Data Aggregation, Processing, Searching and Visualization
> with Open Source Software*****
>
> *http://massivelogdata.com*****
>
>  ****
>
> On 19 July 2013 14:00, Kumar, Deepak8 <deepak8.kumar@citi.com> wrote:****
>
> Hi,****
>
> Could anyone please assist on this?****
>
>  ****
>
> Regards,****
>
> Deepak****
>
>  ****
>
> *From:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Sent:* Monday, July 15, 2013 8:00 AM
> *To:* 'user@flume.apache.org'
> *Cc:* Kumar, Deepak8 [CCC-OT_IT NE]
> *Subject:* How to configure Regex in file names flume 1.3****
>
>  ****
>
> Hi,****
>
> I am using flume 1.3. Could we configure files names as regular
> expressions in flume-conf.properties.****
>
>  ****
>
> e.g.****
>
> agent.sources.source1.type = exec****
>
> agent.sources.source1.command = tail -F /var/log/flume-ng/template.log****
>
> agent.sources.source1.channels = fileChannel****
>
> agent.sources.source1.interceptors = logIntercept****
>
>  ****
>
> Could we have it something like this in flume 1.3****
>
> agent.sources.source1.command = tail -F /var/log/flume-ng/template.*****
>
>  ****
>
> Regards,****
>
> Deepak****
>
>  ****
>
> ** **
>

Mime
View raw message