hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nigel Daley <nda...@yahoo-inc.com>
Subject Re: How to use MultithreadedMapRunner and MapRunner with the same hadoop-site.xml
Date Thu, 25 Jan 2007 17:32:31 GMT

On Jan 25, 2007, at 9:23 AM, Doug Cutting wrote:

> Gu wrote:
>> How can I use in some case MultithreadedMapRunner, and in some case
>> MapRunner for different jobs?
>
> Use JobConf#setMapRunnerClass() on jobs that you want to override  
> the default MapRunner, with, e.g. MultithreadedMapRunner.
>
>> Do I have to use one hadoop-site.xml for one job? But I want to  
>> all jobs in
>> one jobs.jar?
>
> The job.xml is generated from a JobConf is independent of the  
> job.jar, so you can use a single hadoop-site.xml and a single  
> job.jar with multiple jobs whose other JobConf parameters vary.

Note, however, that if you're going to pass the same JobConf instance  
(perhaps with modified attributes) to JobClient.submitJob() within  
one run, then you need to clone it first.  See http:// 
issues.apache.org/jira/browse/HADOOP-701

Nige

Mime
View raw message