hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Fabbri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.
Date Wed, 26 Oct 2016 02:59:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15607226#comment-15607226
] 

Aaron Fabbri commented on HADOOP-13589:
---------------------------------------

My latest patch on HADOOP-13651 has all the integration tests working.  I added a couple of
conditional skips of assertions for things like FS stats where we avoid a round-trip to S3.
 Right now to run this, I manually uncomment this part of hadoop-aws/src/test/resources/core-site.xml:

{code}
<!-- Uncomment these two properties to run integration tests with
       local metadata store (S3Guard).

  <property>
    <name>fs.s3a.metadatastore.authoritative</name>
    <value>true</value>
  </property>
  <property>
    <name>fs.s3a.metadatastore.impl</name>
    <value>org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore</value>
  </property>

  -->
{code}

But we probably want to automate (as you mention) these three cases:

1. Default (null metadata store)
2. LocalMetadataStore, fs.s3a.metadatastore.authoritative=false
3. LocalMetadataStore, fs.s3a.metadatastore.authoritative=true

The downside is runtime.  #1 we will get by default, but maybe we add a profile that runs
#2 then #3?


> S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-13589
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13589
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Chris Nauroth
>
> With S3Guard enabled, S3A must continue to be functionally correct.  The best way to
verify this is to execute our existing S3A integration tests in a mode with S3A enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message