hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ilyal levin <nipponil...@gmail.com>
Subject Re: How to Create an effective chained MapReduce program.
Date Mon, 05 Sep 2011 19:21:44 GMT
Thanks for the reply.
I tried it but it creates a binary file which i can not understand (i need
the result of the first job).
The other thing is how can i use this file in the next chained mapper? i.e
how can i retrieve the keys and the values in the map function?


Ilyal

On Mon, Sep 5, 2011 at 7:41 PM, Joey Echeverria <joey@cloudera.com> wrote:

> Have you tried SequenceFileOutputFormat and SequenceFileInputFormat?
>
> -Joey
>
> On Mon, Sep 5, 2011 at 11:49 AM, ilyal levin <nipponilyal@gmail.com>
> wrote:
> > Hi
> > I'm trying to write a chained mapreduce program. i'm doing so with a
> simple
> > loop where in each iteration i
> > create a job ,execute it and every time the current job's output is the
> next
> > job's input.
> > how can i configure the outputFormat of the current job and the
> inputFormat
> > of the next job so that
> > i will not use the TextInputFormat (TextOutputFormat), because if i do
> use
> > it, i need to parse the input file in the Map function?
> > i.e if possible i want the next job to "consider" the input file as
> > <key,value> and not plain Text.
> > Thanks a lot.
> >
> >
> >
>
>
>
> --
> Joseph Echeverria
> Cloudera, Inc.
> 443.305.9434
>

Mime
View raw message