hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13448) S3Guard: Define MetadataStore interface.
Date Tue, 23 Aug 2016 05:51:20 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Nauroth updated HADOOP-13448:
    Attachment: HADOOP-13448-HADOOP-13345.001.patch

I'm attaching a patch to get the ball rolling.  I approached this by looking at both prototype
patches attached to HADOOP-13345 and trying to distill the required parts of each one into
a single interface.  Here is a summary:

* I named it {{MetadataStore}}, as in Aaron's prototype, not {{ConsistentStore}} as in mine.
 The former is more accurate, because our current design allows for S3Guard to take responsibility
for a variety of concerns, such as caching, instead of just consistency.
* I declared an {{initialize}} method and {{implements Closeable}}, as in my prototype.  I
prefer that {{MetadataStore}} implementations take responsibility for their init/shutdown
* I declared {{delete}} and {{deleteSubtree}} as separate methods as in my prototype rather
than a single method with a recursive {{boolean}} argument as in Aaron's prototype.  I have
a slight preference for the explicit breakdown of distinct methods.  If people strongly prefer
the {{boolean}} argument for closer symmetry with {{FileSystem}}, then I would be willing
to compromise.
* I used the name {{put}} (Aaron's prototype) instead of {{save}} (my prototype).  I have
not included a separate {{putNew}}, because I'd like to explore the possibility that the passed
metadata object is sufficient to describe the {{put}} vs. {{putNew}} use case from Aaron's
prototype patch.
* {{PathMetadata}} is mostly a copy from my prototype in this revision.  I expect before we're
done we'll need to add the {{isFullyCached}} flag to {{DirectoryPathMetadata}} or introduce
a new sub-type for {{FullyCachedDirectoryPathMetadata}}.  I also expect we'll need to add
a sub-type for tombstones to track explicitly that a path was deleted.

This is very much open to suggestions from others, so please let me know your feedback.  This
is also not planned to be a public interface, at least not in the short-term, so we'll have
freedom to evolve it to meet our requirements.

Pre-commit will warn that there are no tests.  Since this is (mostly) inteface definition,
I don't plan to write tests immediately.  Tests written for subsequent S3Guard sub-tasks would
cover this.

> S3Guard: Define MetadataStore interface.
> ----------------------------------------
>                 Key: HADOOP-13448
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13448
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HADOOP-13448-HADOOP-13345.001.patch
> Define the common interface for metadata store operations.  This is the interface that
any metadata back-end must implement in order to integrate with S3Guard.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message