hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Binglin Chang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-4900) Dynamic configuration for task slots on TT
Date Thu, 26 Sep 2013 03:30:10 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Binglin Chang updated MAPREDUCE-4900:
-------------------------------------

    Attachment: MAPREDUCE-4944-4900.jmx.v3.patch

Attach patch for MAPREDUCE-4900 and MAPREDUCE-4944, since it is closely related. Changes:
1. Add new admin protocol DynamicResourceProtocol, which include APIs to list node, get node
information and set node slot by host.
2. intercept TaskTracker's heartbeat to JobTracker, change slot number.
3. Also add related API to JMX.
4. Add a script(bin/update-slots.sh) to batch update slots.

                
> Dynamic configuration for task slots on TT
> ------------------------------------------
>
>                 Key: MAPREDUCE-4900
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4900
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: resourcemanager, tasktracker
>    Affects Versions: 1.1.1
>            Reporter: Junping Du
>            Assignee: Binglin Chang
>         Attachments: MAPREDUCE-4900-demo.patch, MAPREDUCE-4900-demo-with-JMX.patch, MAPREDUCE-4900-demo-with-JMX.v2.patch,
MAPREDUCE-4944-4900.jmx.v3.patch
>
>
> The current Hadoop MRV1 resource management logic assumes per node slot number is static
during the lifetime of the TT process. Allowing run-time configuration on per node slot will
give us finer granularity of resource elasticity. This allows Hadoop workloads to coexist
with other workloads on the same hardware efficiently, whether or not the environment is virtualized.
> For more background or design details of this effort, please refer proposal in HADOOP-9165.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message