hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juwei Shi <shiju...@gmail.com>
Subject how to start the customized Child JVM task process of m/r?
Date Tue, 26 Apr 2011 13:02:45 GMT

As you know, Hadoop MapReduce start child JVM processes to run task. I want
to start the process of m/r task myself thus I can pass some OS level
parameters to the JVM process.

For example, set the CPU affinity of the process:
taskset -c $processor java_process_for_the_m/r_task_command

For another example, bind local memory to the processor running the process
using NUMA
numactl -membind 0 java_process_for_the_m/r_task_command

How could I implement this? Should I change Hadoop script or source code?

Thanks in advanced!

View raw message