Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 15 Apr 2015 01:21:58 +0000 (UTC)
From: "Jing Zhao (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12819957.1428645700000.3689.1429060918814@Atlassian.JIRA>
In-Reply-To: <JIRA.12819957.1428645700000@Atlassian.JIRA>
References: <JIRA.12819957.1428645700000@Atlassian.JIRA>
 <JIRA.12819957.1428645700242@arcas>
Subject: [jira] [Commented] (HDFS-8120) Erasure coding: created util class
 to analyze striped block groups
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-8120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495520#comment-14495520 ] 

Jing Zhao commented on HDFS-8120:
---------------------------------

Thanks Zhe. The latest patch looks pretty good to me. Some nits:
# Let's write the following code using if-else:
{code}
+        boolean wrongSize = storedBlock.getNumBytes() != reported.getNumBytes();
+        if (storedBlock.isStriped()) {
{code}
i.e., 
{code}
boolean wrongSize;
if (storedBlock.isStriped()) {
  // ....
} else {
  // ....
}
{code}
# In {{DFSTestUtil#createStripedFile}}, instead of using null to indicate
  no need to create directory and EC zone, it may be better to use an additional
  parameter {{toMkdir}}.
# Nit: need to remove 2 spaces before "@Test".
{code}
-  //  @Test
+    @Test
   public void TestFileMoreThanABlockGroup2() throws IOException {
{code}

Besides, {{TestAddStripedBlocks}} failed with some editlog corruption. Did some debugging, I can see at least two issues:
# DFSStripedOutputStream tries to send out an empty packet to close the block even if writing 0 bytes
# Because of the above bug, NN tries to close the file. This exposes another bug in {{BlockInfoStriped}}, which writes its data/parity block numbers into the close editlog but do not read them while loading.

I will create another jira to fix this.

> Erasure coding: created util class to analyze striped block groups
> ------------------------------------------------------------------
>
>                 Key: HDFS-8120
>                 URL: https://issues.apache.org/jira/browse/HDFS-8120
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-8120.000.patch, HDFS-8120.001.patch, HDFS-8120.002.patch, HDFS-8120.003.patch, HDFS-8120.004.patch
>
>
> The patch adds logic of calculating size of individual blocks in a striped block group.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)