hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-6678) Remove FileContext#isFile, isDirectory and exists
Date Thu, 15 Apr 2010 21:58:50 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eli Collins updated HADOOP-6678:
--------------------------------

    Summary: Remove FileContext#isFile, isDirectory and exists  (was: Propose some changes
to FileContext)

> Remove FileContext#isFile, isDirectory and exists
> -------------------------------------------------
>
>                 Key: HADOOP-6678
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6678
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Hairong Kuang
>            Assignee: Eli Collins
>             Fix For: 0.21.0, 0.22.0
>
>         Attachments: hadoop-6678-1.patch
>
>
> # Add a method  Iterator<FileStatus> listStatus(Path), which allows HDFS client
not to have the whole listing in the memory, benefit more from the iterative listing added
in HDFS-985. Move the current FileStatus[] listStatus(Path) to be a utility method.
> # Remove methods isFile(Path), isDirectory(Path), and exists.
> All these methods are implemented by calling getFileStatus(Path).But most users are not
aware of this. They would write code as below: 
> {code}
>   FileContext fc = ..;
>   if (fc.exists(path)) {
>     if (fc.isFile(path)) {
>      ...
>     } else {
>     ...
>     }
>   }
> {code}
> The above code adds unnecessary getFileInfo RPC to NameNode. In our production clusters,
we often see that the number of getFileStatus calls is multiple times of the open calls. If
we remove isFile, isDirectory, and exists from FileContext, users have to explicitly call
getFileStatus first, it is more likely that they will write more efficient code as follow:
> {code}
>   FileContext fc = ...;
>   FileStatus fstatus = fc.getFileStatus(path);
>   if (fstatus.isFile() {
>     ...
>   } else {
>     ...
>   }
> {code}

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message