hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-7878) API - expose an unique file identifier
Date Thu, 01 Sep 2016 00:14:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453828#comment-15453828
] 

Sergey Shelukhin edited comment on HDFS-7878 at 9/1/16 12:13 AM:
-----------------------------------------------------------------

One idea behind open(long/InodeId) is to be able to open files consistently; e.g. for partial
caching (one needs to be sure that the cached data and the data read from FS are for the same
file, guarding against overwrites). File ID is easy to propagate between different readers
for this purpose, but it seems that FileStatus would be rather inconvenient. It forces the
caller who is dealing with the FS to get the status by name first (which also only works if
the name is known; in our case we do know the name) and verify that fileId is consistent.
Is it possible to keep both APIs?


was (Author: sershe):
One idea behind open(long/InodeId) is to be able to open files consistently; e.g. for partial
caching (one needs to be sure that the cached data and the data read from FS are for the same
file, guarding against overwrites). File ID is easy to propagate between different readers
for this purpose, but it seems that FileStatus would be rather inconvenient. It forces the
caller to get the status by name first (which also only works if the name is known; in our
case we do know the name) and verify that fileId is consistent.
Is it possible to keep both APIs?

> API - expose an unique file identifier
> --------------------------------------
>
>                 Key: HDFS-7878
>                 URL: https://issues.apache.org/jira/browse/HDFS-7878
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>              Labels: BB2015-05-TBR
>         Attachments: HDFS-7878.01.patch, HDFS-7878.02.patch, HDFS-7878.03.patch, HDFS-7878.04.patch,
HDFS-7878.05.patch, HDFS-7878.06.patch, HDFS-7878.patch
>
>
> See HDFS-487.
> Even though that is resolved as duplicate, the ID is actually not exposed by the JIRA
it supposedly duplicates.
> INode ID for the file should be easy to expose; alternatively ID could be derived from
block IDs, to account for appends...
> This is useful e.g. for cache key by file, to make sure cache stays correct when file
is overwritten.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message