hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-3150) Allow TT to run children with an elevated oom_adj score
Date Thu, 06 Oct 2011 22:07:30 GMT
Allow TT to run children with an elevated oom_adj score
-------------------------------------------------------

                 Key: MAPREDUCE-3150
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3150
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: mrv2, task-controller
    Affects Versions: 0.20.206.0, 0.23.0
            Reporter: Todd Lipcon


Some users of hadoop have run into issues where memory on the machines gets oversubscribed
for various reasons. When this happens, the machines enter swap, causing things like timeouts,
HBase aborts, etc. One mitigation strategy among many is to run the machines without swap,
and allow the linux OOM killer to kill tasks. However, this is dangerous if the OOM killer
might kill the TT, RS, DN, etc. We can set the {{oom_adj}} value in proc for the MR children
in order to encourage the oom killer to kill the right thing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message