hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre ANCELOT <pierre...@gmail.com>
Subject Dynamically set mapred.tasktracker.map.tasks.maximum from inside a job.
Date Wed, 30 Jun 2010 09:35:14 GMT
Hi everyone :)
There's something I'm probably doing wrong but I can't seem to figure out
what.
I have two hadoop programs running one after the other.
This is done because they don't have the same needs in term of processor in
memory, so by separating them I optimize each task better.
Fact is, I need for the first job on every node
mapred.tasktracker.map.tasks.maximum set to 12.
For the second task, I need it to be set to 20.
so by default I set it to 12 and in the second job's code, I set this:

        Configuration hadoopConfiguration = new Configuration();
        hadoopConfiguration.setInt("mapred.tasktracker.map.tasks.maximum",
20);

But when running the job, instead of having the 20 tasks on each node as
expected, I have 12....
Any idea please?

Thank you.
Pierre.


-- 
http://www.neko-consulting.com
Ego sum quis ego servo
"Je suis ce que je prot├Ęge"
"I am what I protect"

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message