Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9A12E17E11 for ; Mon, 30 Mar 2015 11:44:04 +0000 (UTC) Received: (qmail 21968 invoked by uid 500); 30 Mar 2015 11:43:53 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 21911 invoked by uid 500); 30 Mar 2015 11:43:53 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 21889 invoked by uid 99); 30 Mar 2015 11:43:53 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Mar 2015 11:43:53 +0000 Date: Mon, 30 Mar 2015 11:43:53 +0000 (UTC) From: "Kai Zheng (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HDFS-7679) Unifying erasure coder work, block reader and block writer across client and DataNode MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-7679: ---------------------------- Summary: Unifying erasure coder work, block reader and block writer across client and DataNode (was: EC encode/decode framework) > Unifying erasure coder work, block reader and block writer across client and DataNode > ------------------------------------------------------------------------------------- > > Key: HDFS-7679 > URL: https://issues.apache.org/jira/browse/HDFS-7679 > Project: Hadoop HDFS > Issue Type: Sub-task > Reporter: Li Bo > Assignee: Li Bo > Attachments: ECEncodeDecodeFramework.patch > > > EC tasks such as client striping write, encode replicated file to EC style, transform block layout from contiguous to striping, reconstruct some corrupt blocks, etc, have similar behavior. They all read data from client/BlockGroup, encode/decode them, and write the results to some datanodes. So we can use a unified framework to handle all these tasks. We can use different BlockGroupReader, Coder(ECEncoder/ECDecoder) and BlockWriter to handle different EC tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)