hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "GAO Rui (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files
Date Mon, 04 Apr 2016 00:47:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223571#comment-15223571

GAO Rui commented on HDFS-7661:

Hi [~zhz].

The reason of using two versions is we need to keep the latest flushed parity cells and the
current flushed cells, in case of the current flush failing. We could still keep the safety
of latest flushed datas even if the current flush operation failed. So the minimum request
is using two partial parity cell files. 

I think using two versions in Parity DNS could limit both the writing and reading operations
changes to mainly source code of datanode. For writing/reading client, only minor changes
need to be implemented. While, storing data cells on parity DNs need totally different logical
for writing/reading client implementation. 

> [umbrella] support hflush and hsync for erasure coded files
> -----------------------------------------------------------
>                 Key: HDFS-7661
>                 URL: https://issues.apache.org/jira/browse/HDFS-7661
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: erasure-coding
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: GAO Rui
>         Attachments: EC-file-flush-and-sync-steps-plan-2015-12-01.png, HDFS-7661-unitTest-wip-trunk.patch,
HDFS-7661-wip.01.patch, HDFS-EC-file-flush-sync-design-v20160323.pdf, HDFS-EC-file-flush-sync-design-version1.1.pdf
> We also need to support hflush/hsync and visible length. 

This message was sent by Atlassian JIRA

View raw message