Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 942E4200D20 for ; Tue, 17 Oct 2017 15:19:04 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 91F611609EB; Tue, 17 Oct 2017 13:19:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D87021609DE for ; Tue, 17 Oct 2017 15:19:03 +0200 (CEST) Received: (qmail 90700 invoked by uid 500); 17 Oct 2017 13:19:03 -0000 Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list mapreduce-issues@hadoop.apache.org Received: (qmail 90689 invoked by uid 99); 17 Oct 2017 13:19:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Oct 2017 13:19:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 4A0BF1A3161 for ; Tue, 17 Oct 2017 13:19:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id WQ38FB-eMzue for ; Tue, 17 Oct 2017 13:19:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 320EE5FAC9 for ; Tue, 17 Oct 2017 13:19:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6B557E00B3 for ; Tue, 17 Oct 2017 13:19:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 13B4D24360 for ; Tue, 17 Oct 2017 13:19:00 +0000 (UTC) Date: Tue, 17 Oct 2017 13:19:00 +0000 (UTC) From: "Gergo Repas (JIRA)" To: mapreduce-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (MAPREDUCE-6984) MR AM to clean up temporary files from previous attempt MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 17 Oct 2017 13:19:04 -0000 [ https://issues.apache.org/jira/browse/MAPREDUCE-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gergo Repas updated MAPREDUCE-6984: ----------------------------------- Attachment: MAPREDUCE-6984.001.patch > MR AM to clean up temporary files from previous attempt > ------------------------------------------------------- > > Key: MAPREDUCE-6984 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6984 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: applicationmaster > Affects Versions: 3.0.0-beta1 > Reporter: Gergo Repas > Attachments: MAPREDUCE-6984.000.patch, MAPREDUCE-6984.001.patch > > > When the MR AM restarts, the {outputDir}/_temporary/{appAttemptNumber} directory remains on HDFS, even though this directory is not used during the next attempt if the restart has been done without recovery. So if recovery is not used for the AM restart, then the deletion of this directory can be done earlier (at the start of the next attempt). The benefit is that more free HDFS space is available for the next attempt. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org