Return-Path: X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8E57E18E73 for ; Tue, 25 Aug 2015 17:12:30 +0000 (UTC) Received: (qmail 74821 invoked by uid 500); 25 Aug 2015 17:12:14 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 74561 invoked by uid 500); 25 Aug 2015 17:12:14 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 72802 invoked by uid 99); 25 Aug 2015 17:12:13 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Aug 2015 17:12:13 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 852BEE6835; Tue, 25 Aug 2015 17:12:13 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: wangda@apache.org To: common-commits@hadoop.apache.org Date: Tue, 25 Aug 2015 17:12:44 -0000 Message-Id: In-Reply-To: <7b330ad91dce4f1eb47892a0a40f3933@git.apache.org> References: <7b330ad91dce4f1eb47892a0a40f3933@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [33/42] hadoop git commit: HADOOP-12352. Delay in checkpointing Trash can leave trash for 2 intervals before deleting. Contributed by Casey Brotherton. HADOOP-12352. Delay in checkpointing Trash can leave trash for 2 intervals before deleting. Contributed by Casey Brotherton. Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/af787678 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/af787678 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/af787678 Branch: refs/heads/YARN-1197 Commit: af78767870b8296886c03f8be24cf13a4e2bd4b0 Parents: 57c7ae1 Author: Harsh J Authored: Tue Aug 25 11:21:19 2015 +0530 Committer: Harsh J Committed: Tue Aug 25 11:21:19 2015 +0530 ---------------------------------------------------------------------- hadoop-common-project/hadoop-common/CHANGES.txt | 3 +++ .../java/org/apache/hadoop/fs/TrashPolicyDefault.java | 11 +++++++++-- 2 files changed, 12 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/af787678/hadoop-common-project/hadoop-common/CHANGES.txt ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/CHANGES.txt b/hadoop-common-project/hadoop-common/CHANGES.txt index 4250fc3..0ec4ed6 100644 --- a/hadoop-common-project/hadoop-common/CHANGES.txt +++ b/hadoop-common-project/hadoop-common/CHANGES.txt @@ -796,6 +796,9 @@ Release 2.8.0 - UNRELEASED BUG FIXES + HADOOP-12352. Delay in checkpointing Trash can leave trash for 2 intervals + before deleting (Casey Brotherton via harsh) + HADOOP-11802: DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe) http://git-wip-us.apache.org/repos/asf/hadoop/blob/af787678/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java index 88aeab5..1ed8a46 100644 --- a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java +++ b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java @@ -161,12 +161,19 @@ public class TrashPolicyDefault extends TrashPolicy { @SuppressWarnings("deprecation") @Override public void createCheckpoint() throws IOException { + createCheckpoint(new Date()); + } + + @SuppressWarnings("deprecation") + public void createCheckpoint(Date date) throws IOException { + if (!fs.exists(current)) // no trash, no checkpoint return; Path checkpointBase; synchronized (CHECKPOINT) { - checkpointBase = new Path(trash, CHECKPOINT.format(new Date())); + checkpointBase = new Path(trash, CHECKPOINT.format(date)); + } Path checkpoint = checkpointBase; @@ -287,7 +294,7 @@ public class TrashPolicyDefault extends TrashPolicy { TrashPolicyDefault trash = new TrashPolicyDefault( fs, home.getPath(), conf); trash.deleteCheckpoint(); - trash.createCheckpoint(); + trash.createCheckpoint(new Date(now)); } catch (IOException e) { LOG.warn("Trash caught: "+e+". Skipping "+home.getPath()+"."); }