hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4439) Cleanup memory related resource management
Date Fri, 17 Oct 2008 11:23:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640501#action_12640501

Hemanth Yamijala commented on HADOOP-4439:

bq. If a job has not specified any memory requirements, this variable would be set to the
job's configuration, maybe via the Task object.

This value is a cluster configuration that should be the same across all task trackers. Hence,
we were trying to see if this can be configured in the job tracker and passed on to the task
trackers. The only way I could find to do it is to change the {{Task}} object to hold this
variable and write it via RPC when it is transferred to the tasktrackers.

The other option is to simplify, and say that all tasktrackers will be configured with this
value. This has a chance that there could be errors in configuration if the admin configures
different values for different TTs. But, in an offline discussion with Devaraj, we felt it
was simpler to do it this way, rather than introduce a new field in the {{Task}} object. I
am OK with this approach. Any objections ?

> Cleanup memory related resource management
> ------------------------------------------
>                 Key: HADOOP-4439
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4439
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>            Priority: Blocker
>             Fix For: 0.19.0
> HADOOP-3759 and HADOOP-3581 introduced memory based resource management. This JIRA is
to cleanup certain aspects of the two issues that came up while doing HADOOP-4035, which is
filed to support memory based scheduling 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message