Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D9F4D200D21 for ; Mon, 2 Oct 2017 01:49:04 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id D86281609EF; Sun, 1 Oct 2017 23:49:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 2A5B31609C8 for ; Mon, 2 Oct 2017 01:49:04 +0200 (CEST) Received: (qmail 51002 invoked by uid 500); 1 Oct 2017 23:49:03 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 50989 invoked by uid 99); 1 Oct 2017 23:49:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Oct 2017 23:49:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id C7F3C18E1E0 for ; Sun, 1 Oct 2017 23:49:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id VAZyFU7jDdsz for ; Sun, 1 Oct 2017 23:49:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 6806560FC8 for ; Sun, 1 Oct 2017 23:49:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id E310BE0F1C for ; Sun, 1 Oct 2017 23:49:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 4F153242C1 for ; Sun, 1 Oct 2017 23:49:00 +0000 (UTC) Date: Sun, 1 Oct 2017 23:49:00 +0000 (UTC) From: "Hudson (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HADOOP-3) Output directories are not cleaned up before the reduces run MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Sun, 01 Oct 2017 23:49:05 -0000 [ https://issues.apache.org/jira/browse/HADOOP-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16187593#comment-16187593 ] Hudson commented on HADOOP-3: ----------------------------- FAILURE: Integrated in Jenkins build HBase-2.0 #609 (See [https://builds.apache.org/job/HBase-2.0/609/]) HBASE-17441 Fix invalid quoting around hadoop-3 build in yetus (elserj: rev 8c6ed571bdbbd3eb5fe38aa0833124ca75c19cfa) * (edit) dev-support/hbase-personality.sh > Output directories are not cleaned up before the reduces run > ------------------------------------------------------------ > > Key: HADOOP-3 > URL: https://issues.apache.org/jira/browse/HADOOP-3 > Project: Hadoop Common > Issue Type: Bug > Affects Versions: 0.1.0 > Reporter: Owen O'Malley > Assignee: Owen O'Malley > Priority: Minor > Fix For: 0.1.0 > > Attachments: clean-out-dir.patch, noclobber.patch > > > The output directory for the reduces is not cleaned up and therefore if you can see left overs from previous runs, if they had more reduces. For example, if you run the application once with reduces=10 and then rerun with reduces=8, your output directory will have frag00000 to frag00009 with the first 8 fragments from the second run and the last 2 fragments from the first run. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: common-issues-help@hadoop.apache.org