hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9565) Add a Blobstore interface to add to blobstore FileSystems
Date Tue, 09 May 2017 15:33:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002879#comment-16002879
] 

Steve Loughran commented on HADOOP-9565:
----------------------------------------

Note how this is a query-only API, allowing for dynamic probes of FS features, rather than
static "does this FS implement a specific interface", and because its done in the base FSClass,
no need to add a new interface to cast an FS to to look for it. We just need to make the defaults
valid for the filesystems, which we can do by having core-default/xml define the default FS
behaviour and with every FS we manage defining their own.

What about external filesystems? Well, the default resolver looks for a resource called {{contract/fs-$SCHEMA-features.xml}}.
If the people implementing filesystems copy their contract test XML file into that location
in the production JAR, it will be picked up automatically. This allows the authors to update
their JARs and have the capabilities be visible on Hadoop implementations with the new API,
but still load/run against the old one.

see also: {{StreamCapabilities}}

> Add a Blobstore interface to add to blobstore FileSystems
> ---------------------------------------------------------
>
>                 Key: HADOOP-9565
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9565
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs, fs/s3, fs/swift
>    Affects Versions: 2.6.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-9565-001.patch, HADOOP-9565-002.patch, HADOOP-9565-003.patch,
HADOOP-9565-004.patch, HADOOP-9565-005.patch, HADOOP-9565-006.patch, HADOOP-9565-008.patch,
HADOOP-9565-010.patch, HADOOP-9565-branch-2-007.patch
>
>
> We can make the fact that some {{FileSystem}} implementations are really blobstores,
with different atomicity and consistency guarantees, by adding a {{Blobstore}} interface to
add to them. 
> This could also be a place to add a {{Copy(Path,Path)}} method, assuming that all blobstores
implement at server-side copy operation as a substitute for rename.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message