hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Merto Mertek <masmer...@gmail.com>
Subject Configure hadoop scheduler
Date Tue, 20 Dec 2011 16:51:04 GMT

I am having problems with changing the default hadoop scheduler (i assume
that the default scheduler is a FIFO scheduler).

I am following the guide located in hadoop/docs directory however I am not
able to run it.  Link for scheduling administration returns an http error
404 ( http://localhost:50030/scheduler ). In the UI under scheduling
information I can see only one queue named "default". mapred-site.xml file
is accessible because when changing a port for a jobtracker I can see a
daemon running with a changed port. Variable $HADOOP_CONFIG_DIR was added
to .bashrc, however that did not solve the problem. I tried to rebuild
hadoop, manualy place the fair scheduler jar in hadoop/lib and changed the
hadoop classpath in hadoop-env.sh to point to the lib folder, but without
success. The only info of the scheduler that is seen in the jobtracker log
is the folowing info:

Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT,
> limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)

I am working on this several days and running out of ideas... I am
wondering how to fix it and where to check currently active scheduler

Config files:
mapred-site.xml <http://pastebin.com/HmDfWqE1>
allocation.xml <http://pastebin.com/Uexq7uHV>
Tried versions: 0.20.203 and 204

Thank you

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message