hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7269) S3 Native should allow customizable file meta-data (headers)
Date Fri, 13 May 2011 21:42:47 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033323#comment-13033323
] 

Todd Lipcon commented on HADOOP-7269:
-------------------------------------

This opens up scope creep that makes me a little bit nervous. If we add the ability to set
metadata in create(), then we probably also need the ability to query metadata, modify metadata
on existing files, etc, which is all of a sudden a lot of code.

There's a trade-off between least-common-denominator APIs (sucks for everyone) vs support-all-FS-features
(sucks for compatibility). I think this might be going too far towards the latter.

Would it make sense to add these functions directly to the S3 filesystems but not to the overall
API? Then if a user really needs to code specifically against S3, they can down-cast to S3FS
and do what they like.

> S3 Native should allow customizable file meta-data (headers)
> ------------------------------------------------------------
>
>                 Key: HADOOP-7269
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7269
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs/s3
>            Reporter: Nicholas Telford
>            Assignee: Nicholas Telford
>            Priority: Minor
>         Attachments: HADOOP-7269-S3-metadata-001.diff, HADOOP-7269-S3-metadata-002.diff
>
>
> The S3 Native FileSystem currently writes all files with a set of default headers:
>  * Content-Type: binary/octet-stream
>  * Content-Length: <computed from object size>
>  * Content-MD5: <computed from object data>
> This is a good start, however many applications would benefit from the ability to customize
(for example) the Content-Type and Expires headers for the file. Ideally the implementation
should be abstract enough to customize all of the available S3 headers and provide a facility
for other FileSystems to specify optional file metadata.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message