hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-922) Automatic configuration of number of map and reduce slots based on available resources for dealing with heterogenous clusters
Date Tue, 16 Mar 2010 22:02:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12846157#action_12846157
] 

Steve Loughran commented on MAPREDUCE-922:
------------------------------------------

MAPREDUCE-1603 could provide a way to do this customisation without having to have logic built
into hadoop itself, logic that may date fast and depend upon the specific workload. 

> Automatic configuration of number of map and reduce slots based on available resources
for dealing with heterogenous clusters
> -----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-922
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-922
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: tasktracker
>    Affects Versions: 0.22.0
>            Reporter: Arun C Murthy
>
> Currently the number of map and reduce slots have to be configured manually, which is
reasonable for homogeneous clusters. However, as the clusters start to change over time it
becomes rather painful to administer and configure. We should start thinking along the direction
of auto-magically configuring the slots based on available resources on the node such as RAM,
CPU, disk etc. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message