hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "GAO Rui (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7661) Support read when a EC file is being written
Date Tue, 01 Dec 2015 12:15:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15033589#comment-15033589
] 

GAO Rui commented on HDFS-7661:
-------------------------------

Thank you very much, [~szetszwo].  

I have made a draft flush/sync steps plan for EC files [^EC-file-flush-and-sync-steps-plan-2015-12-01.png].

The plan is based on the ideas of {{DFSStripedOutputStream#closeImpl( )}} and {{DFSOutputStream#flushOrSync(
)}}.  The steps in yellow ovals is the planned EC file flush/sync steps.  

The most different part between flush/sync steps with {{DFSStripedOutputStream#closeImpl(
)}} is in *step 1.2*  and *step 3*. 

In *step 1.2*, we plan to send an empty "signal" packet to all DN.
In *step 3*, we do not close streamers. And we update visible length if required.

I am aware of that, besides the plan steps, there are a lot of other details should be taken
care of. Before diving into the implementing codes in detail. I want to confirm the big picture
of the process with you. Please feel free to comment and conduct any concerns you have. 

Thank you very much.

> Support read when a EC file is being written
> --------------------------------------------
>
>                 Key: HDFS-7661
>                 URL: https://issues.apache.org/jira/browse/HDFS-7661
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: GAO Rui
>         Attachments: EC-file-flush-and-sync-steps-plan-2015-12-01.png, HDFS-7661-unitTest-wip-trunk.patch
>
>
> We also need to support hflush/hsync and visible length. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message