hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: 0.20.2 : Running Chained jobs using JobControl
Date Sun, 02 Jan 2011 11:06:05 GMT
Although for complex workflows one should checkout Oozie or Azkaban.

On Sun, Jan 2, 2011 at 1:55 PM, Hari Sreekumar <hsreekumar@clickable.com> wrote:
> Can't we run chained jobs like this?
> bool j1 = job1.waitForCompletion(..) ;
> if(b1) job2.waitForCompletion(..) ;
> and setting up the jobs such that job1's output dir is job2's input dir?
> Thanks,
> Hari

Yes, this could work for simple success/failure based chaining
(although it makes the driver code look a tad messy?).

This is what JobControl is aiming to provide from within Hadoop
libraries itself. Plus the ability to have more control on the
dependent, waiting jobs.

Harsh J

View raw message