Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 79356188BB for ; Wed, 6 May 2015 23:39:02 +0000 (UTC) Received: (qmail 88415 invoked by uid 500); 6 May 2015 23:39:02 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 88349 invoked by uid 500); 6 May 2015 23:39:02 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 88338 invoked by uid 99); 6 May 2015 23:39:02 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 May 2015 23:39:02 +0000 Date: Wed, 6 May 2015 23:39:02 +0000 (UTC) From: "Zhe Zhang (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HDFS-7678) Erasure coding: DFSInputStream with decode functionality (pread) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531692#comment-14531692 ] Zhe Zhang commented on HDFS-7678: --------------------------------- Thanks for the feedback Jing. It turns out that the test will fail because because on the codec level we can only decode with all parity blocks present. HADOOP-11566 will address it. To reduce dependency on codec and output stream, I will keep adding tests to {{TestReadStripedFile}} (would be great if you can review HDFS-8334 first) instead of the end-to-end test. > Erasure coding: DFSInputStream with decode functionality (pread) > ---------------------------------------------------------------- > > Key: HDFS-7678 > URL: https://issues.apache.org/jira/browse/HDFS-7678 > Project: Hadoop HDFS > Issue Type: Sub-task > Affects Versions: HDFS-7285 > Reporter: Li Bo > Assignee: Zhe Zhang > Attachments: BlockGroupReader.patch, HDFS-7678-HDFS-7285.002.patch, HDFS-7678-HDFS-7285.003.patch, HDFS-7678-HDFS-7285.004.patch, HDFS-7678-HDFS-7285.005.patch, HDFS-7678-HDFS-7285.006.patch, HDFS-7678-HDFS-7285.007.patch, HDFS-7678-HDFS-7285.008.patch, HDFS-7678-HDFS-7285.009.patch, HDFS-7678-HDFS-7285.010.patch, HDFS-7678.000.patch, HDFS-7678.001.patch > > > A block group reader will read data from BlockGroup no matter in striping layout or contiguous layout. The corrupt blocks can be known before reading(told by namenode), or just be found during reading. The block group reader needs to do decoding work when some blocks are found corrupt. -- This message was sent by Atlassian JIRA (v6.3.4#6332)