hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremy Lewi <jer...@lewi.us>
Subject Re: Creating another map reduce task in a map function
Date Sat, 23 Apr 2011 14:39:19 GMT
Ranjith,

Have you looked at Hadoop Cascading http://www.cascading.org/ ?
Its a framework for creating pipelines for processing data ontop of
Map-Reduce. It sounds like it might work for you. You could have one
operator which used a regular expression to split each line into words,
and then you could have another operator after that which would process
each word.


Jeremy


On Sat, 2011-04-23 at 19:39 +0530, ranjith k wrote:
> Thank u..harsh.
> I have a map function. The inputFormat is line input format. i need to
> run another map reduce task from the map function for each word in
> that line. is it possible???
> 
> 
> On Sat, Apr 23, 2011 at 7:21 PM, Harsh J <harsh@cloudera.com> wrote:
>         It is 'possible' to launch a job from within a Task, yes. But
>         what're
>         you looking to achieve by doing this? I believe there are
>         other good
>         ways to chain jobs, if its that.
>         
>         
>         On Sat, Apr 23, 2011 at 4:33 PM, ranjith k
>         <ranjith42k@gmail.com> wrote:
>         >
>         > hello..
>         >  Is it possible to create an another map reduce task from
>         inside the map
>         > function??
>         > advance thanks...
>         > --
>         > Ranjith k
>         > +918129419842
>         >
>         >
>         
>         
>         
>         
>         --
>         Harsh J
> 
> 
> 
> -- 
> Ranjith k
> +918129419842
> 
> 


Mime
View raw message