hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-303) Make contracts of LocalFileSystem and DistributedFileSystem consistent
Date Fri, 04 Sep 2009 00:25:57 GMT

    [ https://issues.apache.org/jira/browse/HDFS-303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751240#action_12751240

Tom White commented on HDFS-303:

Fixing the API so it doesn't return a boolean to indicate success or failure is a difficult
migration problem for the current FileSystem interface, so I suggest that it goes in the new
FileSystem interface being designed in HADOOP-4952.


bq. When HDFS creates a file under a non-existing path it calls mkdirs() to create missing
directories on the path. I think the rename should follow the same pattern.

Maybe (although this contradicts your later point about keeping the behaviour simple). I'm
OK with either change.


bq. My proposal is to fail rename whenever the target file (or directory) exists.

We could implement this easily by doing the check in the appropriate FileSystem implementation,
so we don't have to worry about the difference between native local filesystems in this case.

bq. I suggest to keep this jira as a base one and create a separate one for each of the points
(same as HDFS-538).

Fine, although we may need to coordinate the testing since it is done by TestLocalFileSystemContract
for the local filesystem (which hasn't been added yet, since its tests don't yet pass).

> Make contracts of LocalFileSystem and DistributedFileSystem consistent
> ----------------------------------------------------------------------
>                 Key: HDFS-303
>                 URL: https://issues.apache.org/jira/browse/HDFS-303
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Tom White
>         Attachments: hadoop-4114.patch, HDFS-303-common-test-case.patch
> There are a number of edge cases that the two file system implementations handle differently.
In particular:
> * When trying to make a directory under an existing file, HDFS throws an IOException
while LocalFileSystem doesn't.
> * The FileSytem#listStatus(Path) method returns null for a non-existent file on HDFS,
while LocalFileSytem returns an empty FileStatus array.
> * When trying to rename a non-existent path, LocalFileSystem throws an IOException, while
HDFS returns false.
> * When renaming a file or directory to a non-existent directory (e.g. /a/b to /c/d, where
/c doesn't exist) LocalFileSystem succeeds (returns true) while HDFS fails (false).
> * When renaming a file (or directory) as an existing file (or directory) LocalFileSystem
succeeds (returns true) while HDFS fails (false).
> We should document the expected behaviour for these cases in FileSystem's javadoc, and
make sure all implementations conform to it.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message