hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-9565) Add a Blobstore interface to add to blobstore FileSystems
Date Sun, 01 Feb 2015 18:15:36 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Steve Loughran updated HADOOP-9565:
-----------------------------------
    Attachment: HADOOP-9565-003.patch

Patch -003

# renamed class to simply {{ObjectStore}}
# upped semantics mask to 64 bits long.
# pulled all semantic bitmasks to a {{StorageSemantics}} interface
# added the semantics of {{DISTINCT_FILES_AND_DIRECTORIES}}, {{WRITE_IN_PROGRESS_VISIBLE}}
and {{WRITE_ON_FLUSH}}.
# marked up s3, s3n and swift. S3 really has a consistency model that is instance-specific;
need to make it something that you can declare in your fs config options if it matters that
much *and* you know what is offered.
# implemented use case of HADOOP-11525  ; handling of {{WRITE_IN_PROGRESS_VISIBLE}} on a copy,
so avoiding a PUT/GET/PUT/DELETE sequence. 

It'd be good to have a couple more uses of this method sorted out, so that we can see how
well it works...someone needs to write an output committer that does the best it can given
the semantics offered by the destination

> Add a Blobstore interface to add to blobstore FileSystems
> ---------------------------------------------------------
>
>                 Key: HADOOP-9565
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9565
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs, fs/s3, fs/swift
>    Affects Versions: 2.6.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-9565-001.patch, HADOOP-9565-002.patch, HADOOP-9565-003.patch
>
>
> We can make the fact that some {{FileSystem}} implementations are really blobstores,
with different atomicity and consistency guarantees, by adding a {{Blobstore}} interface to
add to them. 
> This could also be a place to add a {{Copy(Path,Path)}} method, assuming that all blobstores
implement at server-side copy operation as a substitute for rename.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message