hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Venner <ja...@attributor.com>
Subject Controlling the number of simultanious jobs per machine - 0.15.0
Date Mon, 03 Dec 2007 03:33:56 GMT
We have jobs that require different resources and as such saturate our 
machines at different levels or parallelization.
What we want to do in the driver is set the number of simultaneous jobs 
per node.

        JobClient client = new JobClient();
        Configuration configuration = new Configuration();
        configuration.setInt( "mapred.tasktracker.tasks.maximum", 7);
        JobConf conf = new JobConf(configuration,MergeNewSeenDriver.class);

        System.err.println( "configured maximum tasks is " + conf.get( 
"mapred.tasktracker.tasks.maximum" ));

But this doesn't seem to work. The only success we have had is using 
multithreaded map runner, but then we don't get to run multiple reduces 
at a time on the machines.

Any suggestions?

Mime
View raw message