hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jakob Homan <jho...@yahoo-inc.com>
Subject Re: Creating a job
Date Tue, 11 Aug 2009 23:59:24 GMT
Hey Mithila-
    I would point you to the WordCount example 
(http://hadoop.apache.org/common/docs/current/mapred_tutorial.html) for 
a basic example of how jobs are created by supplying a JobConf to the 
JobClient.  This will submit your conf to the cluster which will create 
and run the job.

The JobControl class is to manager a series of jobs that are dependent 
on each other. Is this a situation you're facing? If not, the job 
submission strategy in the WordCount example should be sufficient.

Regarding threading: Writing multi-thread apps is generally not needed, 
as Hadoop provides parallelization via MapReduce.  However, there is a 
MultithreadedMapper for situations where you may not be maxing out the 
CPU in a specific Mapper.

It sounds like it may be helpful to check out the job submission 

  Let us know if anything is unclear after that.


Jakob Homan

Mithila Nagendra wrote:
> Hello All
> How do I create a Job in Hadoop using Class Job? And how do I run it?
> Generally JobClient.runJob(conf) is used, but the parameter in not of the
> type Job.
> Also How do I use the class JobControl? Can I create Threads in a Hadoop
> (similar to multithreading in JAVA), where different Threads call diffrent
> hadoop jobs? I guess JobControl is connected to all this in some way.
> Thanks for you help
> Mithila Nagendra

View raw message