hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ari Rabkin (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3759) Provide ability to run memory intensive jobs without affecting other running tasks on the nodes
Date Wed, 13 Aug 2008 17:40:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622291#action_12622291

Ari Rabkin commented on HADOOP-3759:

That'll teach me to comment before coffee.

Hemanth --

Yes, I think it makes sense to pass the disk space via the new resourceMap.  I won't have
time to revise the disk space tracking for a week or two.  Shall I open a JIRA, so we don't
lose track?

My only very slight qualm is that I'd rather avoid passing the names, as well as values, of
the various resources we're tracking. As I understand, heartbeats are a bottleneck, so it's
worth being frugal there.  But probably it doesn't make sense to worry about that at this
stage.  We can do serialization magic later if it turns out to be worthwhile.

> Provide ability to run memory intensive jobs without affecting other running tasks on
the nodes
> -----------------------------------------------------------------------------------------------
>                 Key: HADOOP-3759
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3759
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>             Fix For: 0.19.0
>         Attachments: HADOOP-3759.patch, HADOOP-3759.patch, HADOOP-3759.patch
> In HADOOP-3581, we are discussing how to prevent memory intensive tasks from affecting
Hadoop daemons and other tasks running on a node. A related requirement is that users be provided
an ability to run jobs which are memory intensive. The system must provide enough knobs to
allow such jobs to be run while still maintaining the requirements of HADOOP-3581.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message