Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 87943200D20 for ; Tue, 17 Oct 2017 15:18:04 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 857741609EB; Tue, 17 Oct 2017 13:18:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id CBE201609DE for ; Tue, 17 Oct 2017 15:18:03 +0200 (CEST) Received: (qmail 89892 invoked by uid 500); 17 Oct 2017 13:18:03 -0000 Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list mapreduce-issues@hadoop.apache.org Received: (qmail 89881 invoked by uid 99); 17 Oct 2017 13:18:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Oct 2017 13:18:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 400AD1A31B7 for ; Tue, 17 Oct 2017 13:18:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id xBEem2pDL1BH for ; Tue, 17 Oct 2017 13:18:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 1C7DE5FCD3 for ; Tue, 17 Oct 2017 13:18:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 55CACE00B3 for ; Tue, 17 Oct 2017 13:18:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 0FAE624360 for ; Tue, 17 Oct 2017 13:18:00 +0000 (UTC) Date: Tue, 17 Oct 2017 13:18:00 +0000 (UTC) From: "Gergo Repas (JIRA)" To: mapreduce-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (MAPREDUCE-6984) MR AM to clean up temporary files from previous attempt MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 17 Oct 2017 13:18:04 -0000 [ https://issues.apache.org/jira/browse/MAPREDUCE-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gergo Repas updated MAPREDUCE-6984: ----------------------------------- Attachment: (was: MAPREDUCE-6984.001.patch) > MR AM to clean up temporary files from previous attempt > ------------------------------------------------------- > > Key: MAPREDUCE-6984 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6984 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: applicationmaster > Affects Versions: 3.0.0-beta1 > Reporter: Gergo Repas > Attachments: MAPREDUCE-6984.000.patch > > > When the MR AM restarts, the {outputDir}/_temporary/{appAttemptNumber} directory remains on HDFS, even though this directory is not used during the next attempt if the restart has been done without recovery. So if recovery is not used for the AM restart, then the deletion of this directory can be done earlier (at the start of the next attempt). The benefit is that more free HDFS space is available for the next attempt. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org