hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-12681) Fold HdfsLocatedFileStatus into HdfsFileStatus
Date Thu, 16 Nov 2017 04:15:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Douglas updated HDFS-12681:
---------------------------------
    Attachment: HDFS-12681.11.patch

Attaching WIP patch.

This restores the types, but it also preserves a common {{HdfsFileStatus}} API (to avoid changing
internal references excessively, and making the patch un-reviewable). It is now an interface
that is a superset of {{FileStatus}} (will add a unit test for this).

In v11, {{HdfsNamedFileStatus}} is a subtype of {{FileStatus}} while {{HdfsLocatedFileStatus}}
is a subtype of {{LocatedFileStatus}}. Anything user-facing already copied a subset of fields
from these types, so their previous relationship to one another is irrelevant to compatibility
(i.e., the fact that {{HdfsLocatedFileStatus}} is no longer a subtype of {{HdfsFileStatus}}
doesn't matter to user code).

While v10 imposed some changes since the types were not distinct, this only resolves a naming
collision, creating {{getBlockLocations}} and {{getLocatedBlocks}}. It should not be an incompatible
change.

There are two other ways this could be solved.

# Lazily populate block locations by keeping a reference to the {{FileSystem}}. This could
not only interfere with GC, but untangling how that would work with the {{FileSystem}} cache
seems like an unnecessary exercise.
# For HDFS-7878, if {{FileStatus}} (and not {{PathHandle}}) was the token, then the conversion
to the user-facing type could preserve metadata in the opaque blob. This JIRA is required
because {{LocatedFileStatus}} contains only the fields of the base class.

> Fold HdfsLocatedFileStatus into HdfsFileStatus
> ----------------------------------------------
>
>                 Key: HDFS-12681
>                 URL: https://issues.apache.org/jira/browse/HDFS-12681
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>         Attachments: HDFS-12681.00.patch, HDFS-12681.01.patch, HDFS-12681.02.patch, HDFS-12681.03.patch,
HDFS-12681.04.patch, HDFS-12681.05.patch, HDFS-12681.06.patch, HDFS-12681.07.patch, HDFS-12681.08.patch,
HDFS-12681.09.patch, HDFS-12681.10.patch, HDFS-12681.11.patch
>
>
> {{HdfsLocatedFileStatus}} is a subtype of {{HdfsFileStatus}}, but not of {{LocatedFileStatus}}.
Conversion requires copying common fields and shedding unknown data. It would be cleaner and
sufficient for {{HdfsFileStatus}} to extend {{LocatedFileStatus}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message