hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4808) Limit snapshot related methods to DistributedFileSystem
Date Fri, 10 May 2013 22:21:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13654929#comment-13654929

Suresh Srinivas commented on HDFS-4808:

bq. Do you know of any existing or future FileSystem implementations (LocalFileSystem, S3FileSystem,
etc.) which will support snapshots? If they do ever support snapshots, will they support sub-tree
snapshots? (Sub-tree snapshots is not something I'm aware of any other file system supporting.)

FileSystem as I said earlier is a both User API and file system implementors API. Owing it
to being user API, a lot of methods that it currently has will run into the same argument.
Are methods such as getDelegationToken, addDelegationToken, getFileBlockLocation, getBlockSize,
getFileChecksum, checksum verification related methods, getServerDefaults, concat etc. generic?

Perhaps most file systems do snapshots differently. Some do it with a strong notion of volume
level snapshots. They could chose to accordingly implement/tailor the methods in FileSystem.

bq. If literally all other implementers of FileSystem will throw UnsupportedOperationException,
what's the point of putting it in FileSystem at all?
The reason is exactly why the other such methods exist. FileSystem is a user API using which
applications access the file system. I intend to write enhancements (for example another version
of distcp) using snapshot capabilities, especially around snapshot diffs. For this exposing
it using FileSystem APIs is useful.

bq. How about this for a way forward: For now we move the snapshot create/delete APIs to an
HDFS-specific class. If/when another FileSystem implementation shows up which supports a similar
API to the HDFS snapshots API, we can move it to FileSystem at that point.
I am not sure I agree with this. See my arguments above. Our FileSystem interface, how much
ever we pretend it to be generic is heavily influenced by HDFS. One thing that could help
here is, some of the specification work and ideas [~stevel@apache.org] has about ability to
query capabilities etc.
> Limit snapshot related methods to DistributedFileSystem
> -------------------------------------------------------
>                 Key: HDFS-4808
>                 URL: https://issues.apache.org/jira/browse/HDFS-4808
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Suresh Srinivas
> HDFS-2802 introduces snapshots related methods into FileSystem. This jira is for discussing
whether these snapshots related methods are needed in FileSystem or should be limited to only

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message