Return-Path: X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 31A69173C0 for ; Mon, 18 May 2015 17:16:32 +0000 (UTC) Received: (qmail 49546 invoked by uid 500); 18 May 2015 17:16:14 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 48493 invoked by uid 500); 18 May 2015 17:16:13 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 48057 invoked by uid 99); 18 May 2015 17:16:13 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 18 May 2015 17:16:13 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 44ECFE091A; Mon, 18 May 2015 17:16:13 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: zhz@apache.org To: common-commits@hadoop.apache.org Date: Mon, 18 May 2015 17:16:34 -0000 Message-Id: In-Reply-To: <98a1bba7ade845ac808a2f69c0ad96d7@git.apache.org> References: <98a1bba7ade845ac808a2f69c0ad96d7@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [23/50] hadoop git commit: HDFS-8183. Erasure Coding: Improve DFSStripedOutputStream closing of datastreamer threads. Contributed by Rakesh R. HDFS-8183. Erasure Coding: Improve DFSStripedOutputStream closing of datastreamer threads. Contributed by Rakesh R. Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/bf2d0ac7 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/bf2d0ac7 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/bf2d0ac7 Branch: refs/heads/HDFS-7285 Commit: bf2d0ac717768e1cf8592e22fe278b7867381423 Parents: 6dcb9b1 Author: Zhe Zhang Authored: Thu Apr 30 00:13:32 2015 -0700 Committer: Zhe Zhang Committed: Mon May 18 10:01:50 2015 -0700 ---------------------------------------------------------------------- .../hadoop-hdfs/CHANGES-HDFS-EC-7285.txt | 3 +++ .../org/apache/hadoop/hdfs/DFSStripedOutputStream.java | 12 ++++++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/bf2d0ac7/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt index ca60487..3c75152 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt +++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-EC-7285.txt @@ -149,3 +149,6 @@ HDFS-8282. Erasure coding: move striped reading logic to StripedBlockUtil. (Zhe Zhang) + + HDFS-8183. Erasure Coding: Improve DFSStripedOutputStream closing of + datastreamer threads. (Rakesh R via Zhe Zhang) http://git-wip-us.apache.org/repos/asf/hadoop/blob/bf2d0ac7/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java index c930187..5e2a534 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java +++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java @@ -331,18 +331,26 @@ public class DFSStripedOutputStream extends DFSOutputStream { // interrupt datastreamer if force is true @Override protected void closeThreads(boolean force) throws IOException { + int index = 0; + boolean exceptionOccurred = false; for (StripedDataStreamer streamer : streamers) { try { streamer.close(force); streamer.join(); streamer.closeSocket(); - } catch (InterruptedException e) { - throw new IOException("Failed to shutdown streamer"); + } catch (InterruptedException | IOException e) { + DFSClient.LOG.error("Failed to shutdown streamer: name=" + + streamer.getName() + ", index=" + index + ", file=" + src, e); + exceptionOccurred = true; } finally { streamer.setSocketToNull(); setClosed(); + index++; } } + if (exceptionOccurred) { + throw new IOException("Failed to shutdown streamer"); + } } /**