Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C10D4189B3 for ; Wed, 29 Apr 2015 20:04:07 +0000 (UTC) Received: (qmail 73232 invoked by uid 500); 29 Apr 2015 20:04:07 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 73059 invoked by uid 500); 29 Apr 2015 20:04:07 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 73042 invoked by uid 99); 29 Apr 2015 20:04:07 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Apr 2015 20:04:07 +0000 Date: Wed, 29 Apr 2015 20:04:07 +0000 (UTC) From: "Jing Zhao (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (HDFS-8272) Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520095#comment-14520095 ] Jing Zhao edited comment on HDFS-8272 at 4/29/15 8:03 PM: ---------------------------------------------------------- Thanks for the review, Zhe and Yi! I just merged HDFS-8280 into HDFS-7285 branch. Upload the patch excluding the changes in DFSInputStream. Also fix the bug pointed out by Yi. About the {{seekToBlockSource}}, I think it may be better to remove it: # With decoding functionality we do not need to spend more time trying our luck on the same DN. # Currently calling {{seekToBlockSource}} will cause all the current block readers to be closed (since {{blockSeekTo}} will call {{closeCurrentBlockReaders}}). To fix this we need to add extra complexity so as to make sure only one block reader is retried. How about removing it by now and we can add it back if necessary in the future? was (Author: jingzhao): Thanks for the review, Zhe and Yi! I just merged HDFS-8280 into HDFS-7285 branch. Upload the patch excluding the changes in DFSInputStream. Also fix the bug pointed out by Yi. About the {{seekToBlockSource}}, I think it may be better to remove it by now: # With decoding functionality we do not need to spend more time trying our luck on the same DN. # Currently calling {{seekToBlockSource}} will cause all the current block readers to be closed (since {{blockSeekTo}} will call {{closeCurrentBlockReaders}}). To fix this we need to add extra complexity so as to make sure only one block reader is retried. > Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read) > --------------------------------------------------------------------------------- > > Key: HDFS-8272 > URL: https://issues.apache.org/jira/browse/HDFS-8272 > Project: Hadoop HDFS > Issue Type: Sub-task > Reporter: Jing Zhao > Assignee: Jing Zhao > Attachments: HDFS-8272.002.patch, h8272-HDFS-7285.000.patch, h8272-HDFS-7285.001.patch > > > Currently in DFSStripedInputStream the retry logic is still the same with DFSInputStream. More specifically, every failed read will try to search for another source node. And an exception is thrown when no new source node can be identified. This logic is not appropriate for EC inputstream and can be simplified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)