hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "GAO Rui (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7661) Erasure coding: support hflush and hsync
Date Thu, 18 Feb 2016 08:13:18 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15151916#comment-15151916
] 

GAO Rui commented on HDFS-7661:
-------------------------------

[~liuml07], your work separation look good. I have added {{isOverwrite}} in packet header
in the wip patch, like said in design document 2, we could also add {{blockGroutLen}} into
packet header as well. For the data safety issue [~walter.k.su] , [~zhz] and I discussed earlier
in this jira. I prefer to store the overwritten data(the last KBs of partial stripe parity
internal block) into the related .meta file, in case of the failure of the second flush on
the same partial stripe damage the data written before the first flush.   {{two-phase commit
alike method}} is also trying to keep the data data written before the first failure, right?


And, Without a lock in NN, the reader might got two different version of parity blocks when
issue a internal data block recovery, right?



> Erasure coding: support hflush and hsync
> ----------------------------------------
>
>                 Key: HDFS-7661
>                 URL: https://issues.apache.org/jira/browse/HDFS-7661
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: GAO Rui
>         Attachments: EC-file-flush-and-sync-steps-plan-2015-12-01.png, HDFS-7661-unitTest-wip-trunk.patch,
HDFS-EC-file-flush-sync-design-version1.1.pdf, HDFS-EC-file-flush-sync-design-version2.0.pdf
>
>
> We also need to support hflush/hsync and visible length. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message