hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6633) Support reading new data in a being written file until the file is closed
Date Tue, 23 Sep 2014 08:33:35 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vinayakumar B updated HDFS-6633:
--------------------------------
    Attachment: HDFS-6633-001.patch

Attached the patch.

changes:
1. Added 2 new APIs, {{pollNewData() and isFileUnderConstruction()}} to DFSInputStream and
HdfsDataInputStream

2. {{pollNewData()}} should be called after EOF on being-written file.

3. Once it returns true, then can continue reading again.


Tried changing the datatransfer protocol to continue reading from the existing stream itself.
But I was facing problem in BlockSender.

> Support reading new data in a being written file until the file is closed
> -------------------------------------------------------------------------
>
>                 Key: HDFS-6633
>                 URL: https://issues.apache.org/jira/browse/HDFS-6633
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: hdfs-client
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Vinayakumar B
>         Attachments: HDFS-6633-001.patch, h6633_20140707.patch, h6633_20140708.patch
>
>
> When a file is being written, the file length keeps increasing.  If the file is opened
for read, the reader first gets the file length and then read only up to that length.  The
reader will not be able to read the new data written afterward.
> We propose adding a new feature so that readers will be able to read all the data until
the writer closes the file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message