hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cyril Briquet (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-246) Add a method to get file length for Seekable, FSDataInputStream and libhdfs
Date Tue, 03 May 2011 11:43:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028164#comment-13028164
] 

Cyril Briquet commented on HDFS-246:
------------------------------------

Here's another use case:
reading a file that contains an unknown number of UTF-encoded Strings

FSDataInputStream file = openFile();
long len = file.length();
while (file.getPos() < len) {
  String s = file.readUTF();
  processString(s);
}

I'd be glad to learn how to implement this pattern with the current HDFS API,
besides reading the file length through FileSystem.getFileStatus().getLen()


> Add a method to get file length for Seekable, FSDataInputStream and libhdfs
> ---------------------------------------------------------------------------
>
>                 Key: HDFS-246
>                 URL: https://issues.apache.org/jira/browse/HDFS-246
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Qi Liu
>            Assignee: Qi Liu
>         Attachments: HADOOP-5143-2.patch, HADOOP-5143.patch, hadoop.patch
>
>
> When open any seekable file, it should be able to get the length of the file via Seekable
interface, since the seek method should be able to detect seeking beyond the end of file.
Such interface can benefit distributed file systems by saving a network round-trip of FileSystem.getFileStatus(Path).getLen()
for any open file.
> In libhdfs, such interface should also be exposed to make native program taking advantage
of this change.
> I have the changes locally for all FSInputStream concrete classes. The change can be
considered trivial, since some of the FSInputStream classes already have a method named getFileLength(),
or a member field named size/length/end.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message