Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <12645302.1151434230707.JavaMail.jira@brutus>
Date: Tue, 27 Jun 2006 18:50:30 +0000 (GMT+00:00)
From: "Doug Cutting (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Commented: (HADOOP-27) MapRed tries to allocate tasks to
 nodes that have no available disk space
In-Reply-To: <1424617142.1139335680270.JavaMail.jira@ajax.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

    [ http://issues.apache.org/jira/browse/HADOOP-27?page=comments#action_12418085 ] 

Doug Cutting commented on HADOOP-27:
------------------------------------

I think this is a good change to make, but the patch still has a few issues.

I don't think it is sufficient to check if any device has enough room, but rather, one should check that all devices have enough room.  Local file names are hashed to determine which device to store a file on.  Unless that algorithm changes (Configuration.getLocalPath()), we should make sure that all local devices have a minimum amount of space.

Also, the indentation and formatting of the patch is non-standard, using tabs instead of spaces, and one chunk of the patch fails to apply cleanly.

Thanks!

> MapRed tries to allocate tasks to nodes that have no available disk space
> -------------------------------------------------------------------------
>
>          Key: HADOOP-27
>          URL: http://issues.apache.org/jira/browse/HADOOP-27
>      Project: Hadoop
>         Type: Bug

>   Components: mapred
>     Reporter: Mike Cafarella
>  Attachments: minspace_mapredtask.patch
>
>   What it says above.  MapRed TaskTrackers should not offer task service if the local disk
> space is too constrained.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira