hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3960) Snapshot of Being Written Files
Date Fri, 16 Nov 2012 22:51:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499215#comment-13499215
] 

Tsz Wo (Nicholas), SZE commented on HDFS-3960:
----------------------------------------------

BTW, we are going to do the following as a conclusion of the previous meetup.

- Report hflush length to NN via heartbeats from DN as suggested by Suresh and Sanjay.

- Add length information to the sync rpc call from client to NN as suggested by Konstantin.
 Then, application like HBase could coordinate themselves to sync the length before taking
snapshot. We have mentioned _super hflush_ (hflush with an additional rpc call to NN) but
it seems that we could simply use hsync but not adding a new API.
                
> Snapshot of Being Written Files
> -------------------------------
>
>                 Key: HDFS-3960
>                 URL: https://issues.apache.org/jira/browse/HDFS-3960
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: data-node, hdfs client, name-node
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Jing Zhao
>
> Here is a design question: Suppose there is a being written file when a snapshot is being
taken.  What should the length of the file be shown in the snapshot?  In other words, how
to determine the length of being written file when a snapshot is being taken?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message