Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <2101146414.1213947465044.JavaMail.jira@brutus>
Date: Fri, 20 Jun 2008 00:37:45 -0700 (PDT)
From: "Arun C Murthy (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Updated: (HADOOP-3598) Map-Reduce framework needlessly
 creates temporary _${taskid} directories for Maps
In-Reply-To: <1428332829.1213887945131.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HADOOP-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arun C Murthy updated HADOOP-3598:
----------------------------------

    Status: Open  (was: Patch Available)

> Map-Reduce framework needlessly creates temporary _${taskid} directories for Maps
> ---------------------------------------------------------------------------------
>
>                 Key: HADOOP-3598
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3598
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.18.0
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>            Priority: Blocker
>             Fix For: 0.18.0
>
>         Attachments: HADOOP-3598_0_20080619.patch, HADOOP-3598_0_20080619.patch, HADOOP-3598_0_20080619.patch
>
>
> The staging directory for task-outputs (i.e. ${mapred.out.dir}/_temporary/_${taskid}) should only be created when Maps produce output on HDFS, which usually isn't the case. This plays very badly with HDFS quotas and may lead to thousands of temp names in the FS namespace, there-by overhauling the quotas. IAC, it isn't good to needlessly create these directories.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.