Return-Path: 
 <hdfs-dev-return-23707-apmail-hadoop-hdfs-dev-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-hdfs-dev-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-hdfs-dev-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 95CF9187C8
	for <apmail-hadoop-hdfs-dev-archive@minotaur.apache.org>;
 Fri, 19 Feb 2016 03:59:47 +0000 (UTC)
Received: (qmail 39186 invoked by uid 500); 19 Feb 2016 03:54:22 -0000
Delivered-To: apmail-hadoop-hdfs-dev-archive@hadoop.apache.org
Received: (qmail 38711 invoked by uid 500); 19 Feb 2016 03:54:21 -0000
Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:hdfs-dev-help@hadoop.apache.org>
List-Unsubscribe: <mailto:hdfs-dev-unsubscribe@hadoop.apache.org>
List-Post: <mailto:hdfs-dev@hadoop.apache.org>
List-Id: <hdfs-dev.hadoop.apache.org>
Reply-To: hdfs-dev@hadoop.apache.org
Delivered-To: mailing list hdfs-dev@hadoop.apache.org
Received: (qmail 37430 invoked by uid 99); 19 Feb 2016 03:48:21 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 19 Feb 2016 03:48:21 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 1F6BC2C1F6B
	for <hdfs-dev@hadoop.apache.org>; Fri, 19 Feb 2016 03:48:18 +0000 (UTC)
Date: Fri, 19 Feb 2016 03:48:18 +0000 (UTC)
From: "Lin Yiqun (JIRA)" <jira@apache.org>
To: hdfs-dev@hadoop.apache.org
Message-ID: <JIRA.12940375.1455853664000.89947.1455853698124@Atlassian.JIRA>
In-Reply-To: <JIRA.12940375.1455853664000@Atlassian.JIRA>
References: <JIRA.12940375.1455853664000@Atlassian.JIRA>
 <JIRA.12940375.1455853664968@arcas>
Subject: [jira] [Created] (HDFS-9832) Erasure Coding: Improve exception
 handling in ErasureCodingWorker#ReconstructAndTransferBlock
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394

Lin Yiqun created HDFS-9832:
-------------------------------

             Summary: Erasure Coding: Improve exception handling in ErasureCodingWorker#ReconstructAndTransferBlock
                 Key: HDFS-9832
                 URL: https://issues.apache.org/jira/browse/HDFS-9832
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: erasure-coding
            Reporter: Lin Yiqun
            Assignee: Lin Yiqun
            Priority: Minor
             Fix For: 3.0.0


There are two places in {{ErasureCodingWorker#ReconstructAndTransferBlock}} that I think can be improved.
1.In run method, the step3 transfer data will be failed sometimes, and this will cause buffers not be cleared completely, is better to invoke clearBuffer again in finally handling?
{code}
        while (positionInBlock < maxTargetLength) {
          final int toReconstruct = (int) Math.min(
              bufferSize, maxTargetLength - positionInBlock);
          // step1: read from minimum source DNs required for reconstruction.
          // The returned success list is the source DNs we do real read from
          Map<ExtendedBlock, Set<DatanodeInfo>> corruptionMap = new HashMap<>();
          try {
            success = readMinimumStripedData4Reconstruction(success,
                toReconstruct, corruptionMap);
          } finally {
            // report corrupted blocks to NN
            reportCorruptedBlocks(corruptionMap);
          }

          // step2: decode to reconstruct targets
          reconstructTargets(success, targetsStatus, toReconstruct);

          // step3: transfer data
          if (transferData2Targets(targetsStatus) == 0) {
            String error = "Transfer failed for all targets.";
            throw new IOException(error);
          }

          clearBuffers();
          positionInBlock += toReconstruct;
        }
{code}

2.Is better to set null to buffers objects, targetsOutput and socket objects in finally handling code?
{code}
      } finally {
        datanode.decrementXmitsInProgress();
        // close block readers
        for (StripedReader stripedReader : stripedReaders) {
          closeBlockReader(stripedReader.blockReader);
        }
        for (int i = 0; i < targets.length; i++) {
          IOUtils.closeStream(targetOutputStreams[i]);
          IOUtils.closeStream(targetInputStreams[i]);
          IOUtils.closeStream(targetSockets[i]);
        }
      }
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)