chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <eric...@gmail.com>
Subject Re: About CHUKWA-581,Demux do not support user custom reducer package name.
Date Wed, 10 Jul 2013 04:01:14 GMT
Yes that will help.  Please include test case as well.  Thanks Ivy.

regards,
Eric


On Mon, Jul 8, 2013 at 10:42 PM, IvyTang <ivytang0812@gmail.com> wrote:

> This issue has been reported
> https://issues.apache.org/jira/browse/CHUKWA-581.
>
> And i think i can submit a patch later .
>
> Should i open a new jira  or just assign this CHUKWA-581 to myself?
>
> This can implement reducer outside chukwa.
>
> --
> Best regards,
>
> Ivy Tang
>
> On 2013-7-9, at δΈ‹εˆ1:22, Eric Yang <eric818@gmail.com> wrote:
>
> > The recommended change can improve usability to implement parsers outside
> > of Chukwa packages.  +1 on the change.  Do you mind open a jira for this?
> > Thanks
> >
> > regards,
> > Eric
> >
> >
> > On Mon, Jul 8, 2013 at 6:54 AM, 唐 ι’– <ivytang0812@gmail.com> wrote:
> >
> >> We can write our own mapper ,because MapProcessorFactory uses
> >> Class.forName(parserClass).getConstructor().newInstance();
> >> since we config the parserClass in config file.
> >>
> >> But the ReduceProcessorFactory inits  the reducer class using the
> >> classname
> "org.apache.hadoop.chukwa.extraction.demux.processor.reducer."+
> >> reduceType.
> >> So the reducer class must be put in package
> >> org.apache.hadoop.chukwa.extraction.demux.processor.reducer and named
> >> reduceType.
> >>
> >> Why not we put the reducer  class name in config file either?
> >> For exam, the {customdatatype_mapper} means mapper class
> >> ,{customdatatype_reducer} means reducer class.
> >>
> >> This can give chukka more flexibility.
> >>
> >> --
> >> Best regards,
> >>
> >> Ivy Tang
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message