hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7878) API - expose an unique file identifier
Date Thu, 24 Aug 2017 16:55:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140300#comment-16140300

Chris Douglas commented on HDFS-7878:

bq.  the "better" calls.
Fine by me. If this JIRA only added an option to the builder for {{open}}, that would satisfy
most of our use cases.

bq. I'm a bit confused as to what a PathHandle to an as yet uncreated path would be?
I was thinking we could expose the inode ID for the target directory. HDFS doesn't supply
this now, but it could. The FS implementations over object stores might have trouble supporting

bq. doing this in FileContext would be better
Maybe? This came up during the [discussion|https://issues.apache.org/jira/browse/HADOOP-12910?focusedCommentId=15189492&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15189492]
of async renames. We keep suggesting that we push functionality and users to that API, but
it's not happening. I like the idea of making {{FileContext}} into an "advanced" API that
allows clients to reason about consistency, visibility, performance, and durability. But for
the near and foreseeable future, most of us need the "open by file ID" API.

bq. I know I've been doing things underneath FileSystem, but that doesn't mean we should do
more in terms of evolving that API.
I won't ask for consistency, but I will ask for a short path through this particular issue.
HDFS-6984 took a very long detour while I incorporated all the feedback.

> API - expose an unique file identifier
> --------------------------------------
>                 Key: HDFS-7878
>                 URL: https://issues.apache.org/jira/browse/HDFS-7878
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>              Labels: BB2015-05-TBR
>         Attachments: HDFS-7878.01.patch, HDFS-7878.02.patch, HDFS-7878.03.patch, HDFS-7878.04.patch,
HDFS-7878.05.patch, HDFS-7878.06.patch, HDFS-7878.patch
> See HDFS-487.
> Even though that is resolved as duplicate, the ID is actually not exposed by the JIRA
it supposedly duplicates.
> INode ID for the file should be easy to expose; alternatively ID could be derived from
block IDs, to account for appends...
> This is useful e.g. for cache key by file, to make sure cache stays correct when file
is overwritten.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message