Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 2 Dec 2015 01:52:11 +0000 (UTC)
From: "GAO Rui (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12769497.1421976378000.240569.1449021131060@Atlassian.JIRA>
In-Reply-To: <JIRA.12769497.1421976378000@Atlassian.JIRA>
References: <JIRA.12769497.1421976378000@Atlassian.JIRA>
 <JIRA.12769497.1421976378399@arcas>
Subject: [jira] [Commented] (HDFS-7661) Support read when a EC file is being
 written
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15035103#comment-15035103 ] 

GAO Rui commented on HDFS-7661:
-------------------------------

I got the point of our problem :)  

My proposal is we might could define a mark flag like GS for a stripe(6 data cells + 3 parity cells). For example, when flush at p0. All the related cells is marked with gs1. And the visible length stored in NN indicates the file could be read until the flush point p0. Then, the user call flush at p1. We refresh the mark flag of related cells to gs2. For parity cells, we generated new parity cells at flush point p1, and write parity cells to DNs with gs2. After the acks received from DNs, we update the visible length to the flush point p1. At last, we could remove parity cells with gs1. 

I think I should walk through the reader and namenode side and datanode side related codes to learn how being written replication files was read. Could you give clue key points which you think I should pay more attention to?

> Support read when a EC file is being written
> --------------------------------------------
>
>                 Key: HDFS-7661
>                 URL: https://issues.apache.org/jira/browse/HDFS-7661
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: GAO Rui
>         Attachments: EC-file-flush-and-sync-steps-plan-2015-12-01.png, HDFS-7661-unitTest-wip-trunk.patch
>
>
> We also need to support hflush/hsync and visible length. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)