hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: submit multiple hadoop jobs without opening multiple terminals
Date Sun, 22 Apr 2012 17:02:13 GMT
Is your requirement to not have the job launcher program return until
completion? For that you should either edit the java sources to not
waitForCompletion(…) (and just submit()), or run multiple command
lines using a fork and managing them afterwards.

For example you can do:
bin/hadoop jar hadoop-examples-*.jar grep input output 'dfs[a-z.]+' &

And the process should run in the background until termination,
allowing you to run another without needing to open a new terminal.

Is this what you're looking for?

On Sun, Apr 22, 2012 at 9:51 PM, brisk <mylinqiao@gmail.com> wrote:
> Hi,
>
> Does anybody know how to submit multiple hadoop jobs without opening
> multiple terminals? I found one method is to use Job.Submit() in
> ToolRunner.run(),
> but can I use a shell script to submit jobs (with command like "bin/hadoop
> jar hadoop-examples-*.jar grep input output 'dfs[a-z.]+' ") instead of
>  modifying java files/source code,
> especially if I need to run different kinds of jobs and control the
> inter-arrival time?
>
> Thanks,
> Ethan



-- 
Harsh J

Mime
View raw message