hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Yang <lin.yang.ja...@gmail.com>
Subject Re: How to make different mappers execute different processing on a same data ?
Date Wed, 12 Sep 2012 09:06:09 GMT
All right, I got it~Thank you very much.

2012/9/11 Harsh J <harsh@cloudera.com>

> Hey Jason,
>
> While I am not sure on whats the best way to automatically "evaluate"
> during the execution of a job, the MultipleInputs class offers a way
> to run different map implementations within a single job for different
> input paths. You could perhaps leverage that with duplicated (or
> symlinked?) input paths.
>
> Otherwise, perhaps do all the N types of computation in a single map()
> call, and judge the time inside it at the end of all, before emitting?
>
> On Tue, Sep 11, 2012 at 9:03 AM, Jason Yang <lin.yang.jason@gmail.com>
> wrote:
> > Hi, all
> >
> > I've got a question about how to make different mappers execute different
> > processing on a same data?
> >
> > Here is my scenario:
> > I got to process a data, however, there multiple choices to process this
> > data and I have no idea which one is better, so I was thinking that
> maybe I
> > could execute multiple mappers, in which different processing solution is
> > applied, and eventually the best one is chosen according to some
> evaluation
> > functions.
> >
> > But I'm not sure whether this could be done in MapReduce.
> >
> > Any help would be appreciated.
> >
> > --
> > YANG, Lin
> >
>
>
>
> --
> Harsh J
>



-- 
YANG, Lin

Mime
View raw message