hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-538) DistributedFileSystem::listStatus incorrectly returns null for empty result sets
Date Fri, 14 Aug 2009 02:56:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12743058#action_12743058

Jakob Homan commented on HDFS-538:

   I've actually run into an issue with this.  Patch is done, but when running it through
unit tests, it fails {{TestHDFSFileSystemContract}}, which includes a test called {{testListStatusReturnsNullForNonExistentFile()}},
which was added back in HADOOP-930.  This test is evaluating an element of the contract that
seems incorrect, particularly given the FileSystem API documention: "{{@return the statuses
of the files/directories in the given patch}}," which does not mention returning null.  It
would be much better in this situation to just return a zero-length array.  Otherwise, every
call to listStatus has to include an extra null check, rather than just iterating over the
array (which still succeeds with a zero-length array) or actually checking for zero files
via the length of the array equaling zero.

   This is borne out by listStatus' use in HDFS codebase, where there is only one check for
null; the others assume an array will be returned and could potentially NPE at some point.

   The rub is that the S3 FileSystem implementations do return nulls in this situation, so
if we fix the contract, these will have an incompatible change introduced.  I'm not sure how
wide-felt that would be, but it seems to me to be important to fix the semantics sooner than
later.  The other option would be to change the API specification and continue with non-intuitive
behavior, as well as changing HDFS to support it.  Thoughts?

> DistributedFileSystem::listStatus incorrectly returns null for empty result sets
> --------------------------------------------------------------------------------
>                 Key: HDFS-538
>                 URL: https://issues.apache.org/jira/browse/HDFS-538
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Jakob Homan
>            Assignee: Jakob Homan
> Currently the listStatus method returns null if no files match the request.  This differs
from the Checksum/LocalFileSystem implementation, which returns an empty array, and the nontvery-explict
prescription of the FileSystem interface: "{{@return the statuses of the files/directories
in the given patch}}"  It's better to return an empty collection than have to add extra null
checks.  The method should return an empty array.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message