hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rakhi Khatwani <rakhi.khatw...@gmail.com>
Subject Re: Using JobControl in hadoop
Date Mon, 20 Jul 2009 15:45:42 GMT
Hi Tom,

           in that case, can i kill the job by givin some command from the
API?? or i ll have 2 do it frm the command line?

On Mon, Jul 20, 2009 at 8:55 PM, Tom White <tom@cloudera.com> wrote:

> Hi Raakhi,
>
> You can't suspend MapReduce jobs in Hadoop, which is why the
> JobControl API doesn't support job suspension, only the ability to
> kill jobs.
>
> Cheers,
> Tom
>
> On Mon, Jul 20, 2009 at 9:39 AM, Rakhi Khatwani<rakhi.khatwani@gmail.com>
> wrote:
> > Hi,
> >     I have a scenario in which i have a list of 5 jobs. and an event
> > handler for example when triggered, would suspend all the running jobs.
> >
> > but when i use a job control object, jobControl and execute,
> > jobControl.suspend(), it seems tht only the jobControl gets suspended and
> > not the 5 jobs which i wanted.
> >
> >
> > is there any way you can suspend the job in the java program???
> >
> >
> > Regards,
> > Raakhi
> >
> > On Fri, Jul 17, 2009 at 2:48 PM, Tom White <tom@cloudera.com> wrote:
> >
> >> Hi Raakhi,
> >>
> >> JobControl is designed to be run from a new thread:
> >>
> >> Thread t = new Thread(jobControl);
> >> t.start();
> >>
> >> Then you can run a loop to poll for job completion and print out status:
> >>
> >>    String oldStatus = null;
> >>    while (!jobControl.allFinished()) {
> >>      String status = getStatusString(jobControl);
> >>      if (!status.equals(oldStatus)) {
> >>        System.out.println(status);
> >>        oldStatus = status;
> >>      }
> >>      try {
> >>        Thread.sleep(1000);
> >>      } catch (InterruptedException e) {
> >>        // ignore
> >>      }
> >>    }
> >>
> >> Hope this helps.
> >>
> >> Tom
> >>
> >> On Fri, Jul 17, 2009 at 9:10 AM, Rakhi Khatwani<
> rakhi.khatwani@gmail.com>
> >> wrote:
> >> > Hi,
> >> >       I was trying out a map-reduce example using JobControl.
> >> > i create a jobConf conf1 object, add the necessary information
> >> > then i create a job object
> >> > Job job1 = new Job(conf1);
> >> >
> >> > n thn i delare JobControl object as follows:
> >> > JobControl jobControl = new JobControl("JobControl1");
> >> >                  jobControl.addJob(job1);
> >> >                  jobControl.run();
> >> >
> >> >
> >> >
> >> > whn i execute it in the console,
> >> > i get the following output
> >> > 09/07/17 13:10:16 WARN mapred.JobClient: Use GenericOptionsParser for
> >> > parsing the arguments. Applications should implement Tool for the
> same.
> >> > 09/07/17 13:10:16 INFO mapred.FileInputFormat: Total input paths to
> >> process
> >> > : 4
> >> >
> >> >
> >> >
> >> >
> >> > n there is no other output.
> >> > but from the UI i can c that the job has been executed.
> >> > if there any way i can direct the output to the console.
> >> > or is there any way in which while the job is runing, i can continue
> >> > processing from main. (i wanna try suspending/stopping jobs etc).
> >> >
> >> > Regards,
> >> > Raakhi
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message