hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "GAO Rui (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7661) Support read when a EC file is being written
Date Wed, 02 Dec 2015 01:52:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15035103#comment-15035103
] 

GAO Rui commented on HDFS-7661:
-------------------------------

I got the point of our problem :)  

My proposal is we might could define a mark flag like GS for a stripe(6 data cells + 3 parity
cells). For example, when flush at p0. All the related cells is marked with gs1. And the visible
length stored in NN indicates the file could be read until the flush point p0. Then, the user
call flush at p1. We refresh the mark flag of related cells to gs2. For parity cells, we generated
new parity cells at flush point p1, and write parity cells to DNs with gs2. After the acks
received from DNs, we update the visible length to the flush point p1. At last, we could remove
parity cells with gs1. 

I think I should walk through the reader and namenode side and datanode side related codes
to learn how being written replication files was read. Could you give clue key points which
you think I should pay more attention to?

> Support read when a EC file is being written
> --------------------------------------------
>
>                 Key: HDFS-7661
>                 URL: https://issues.apache.org/jira/browse/HDFS-7661
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: GAO Rui
>         Attachments: EC-file-flush-and-sync-steps-plan-2015-12-01.png, HDFS-7661-unitTest-wip-trunk.patch
>
>
> We also need to support hflush/hsync and visible length. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message