flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shumin Wu <shumin...@gmail.com>
Subject Re: Moving from Flume OG to Flume NG
Date Wed, 03 Oct 2012 14:10:55 GMT
Our company has some BI that of course cannot be available in the default
sources provided by flume-ng. So yes, I am using the custom source, in a
way that is pretty much like the code snippet I have pasted in last email.

So far I have found the flume log a good resource to scrub error/exception
informations. You may configure the log4j property file to dump any many
details as you can.

AFAIK, there is no master/node notion in NG any longer. If you want to set
up a distributed cluster, you may take a look at the "setting multi-agent
flow" at
https://people.apache.org/~mpercy/flume/flume-1.2.0-incubating-SNAPSHOT/docs/FlumeUserGuide.html

Good luck!

Shumin

On Wed, Oct 3, 2012 at 3:42 AM, Kumar, Deepak8 <deepak8.kumar@citi.com>wrote:

> Hi Shumin,****
>
> I have gone through the documentation, but it don’t have much. Do you have
> any use case in which you are using CustomSource? As in flume 0.9.4, we do
> have flume master url where we can see the source/sinks/decorators etc. How
> we can verify here whether our custom source is fine & up or have any
> errors?****
>
> ** **
>
> It would be much help if you please guide me a bit.****
>
> ** **
>
> Regards,****
>
> Deepak****
>
> ** **
>
> *From:* Shumin Wu [mailto:shumin.wu@gmail.com]
> *Sent:* Tuesday, September 18, 2012 2:11 PM
> *To:* user@flume.apache.org
> *Subject:* Re: Moving from Flume OG to Flume NG****
>
> ** **
>
> Hi Deepark,
>
> I had hard time switching from OG to NG in the beginning, too. The new way
> in NG to make your custom source would look like this:
>
> public class CustomSource extends AbstractSource implements
> PollableSource, Configurable
>
> or
>
> public class CustomSource extends AbstractSource implements
> EventDrivenSource, Configurable
>
> , depending on your source type.
>
> Brock's link is a good reading resource.
>
> Shumin****
>
> On Tue, Sep 18, 2012 at 11:05 AM, Brock Noland <brock@cloudera.com> wrote:
> ****
>
> Hi,
>
> The flume developer guide as these details:
>
> http://flume.apache.org/FlumeDeveloperGuide.html
>
> Brock****
>
>
> On Tue, Sep 18, 2012 at 1:02 PM, Kumar, Deepak8 <deepak8.kumar@citi.com>
> wrote:
> > Hi,
> > I am trying to switch from Flume 0.94 to Flume NG. In previous version
> we do have the static class EventSource.Base through which we can create
> our own Source or Sink. I don't find anything as such here in flume NG.
> >
> > Could you please guide me how do we create our own Sources here in Flume
> NG?
> >
> > Regards,
> > Deepak
>
>
> ****
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/****
>
> ** **
>

Mime
View raw message