hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13793) s3guard: add inconsistency injection, integration tests
Date Mon, 05 Dec 2016 22:08:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15723516#comment-15723516

Mingliang Liu commented on HADOOP-13793:

	51	  @Override
52	  public void teardown() throws Exception {
53	    super.teardown();
54	  }
56	  @Override
57	  public void nameThread() {
58	    super.nameThread();
59	  }
61	  @Override
62	  protected int getTestTimeoutMillis() {
63	    return super.getTestTimeoutMillis();
64	  }
Can we remove this code lines? Seems unnecessary.

For the inconsistent key delay in ms, I found 500ms are not enough to make the test failure
(when commenting out the {{S3Guard.S3_METADATA_STORE_IMPL}} config key in test). I tried 1000ms+
and found it works. Do you mind increasing the default interval to 1~3 seconds?

In the {{ITestS3GuardListConsistency#createContract}}, this is setting it hard-code. Can we
let use the config files so we can run with DDBMetadataStore?
    conf.setClass(S3Guard.S3_METADATA_STORE_IMPL, LocalMetadataStore.class,


> s3guard: add inconsistency injection, integration tests
> -------------------------------------------------------
>                 Key: HADOOP-13793
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13793
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Aaron Fabbri
>            Assignee: Aaron Fabbri
>         Attachments: HADOOP-13793-HADOOP-13345.001.patch, HADOOP-13793-HADOOP-13345.002.patch
> Many of us share concerns that testing the consistency features of S3Guard will be difficult
if we depend on the rare and unpredictable occurrence of actual inconsistency in S3 to exercise
those code paths.
> I think we should have a mechanism for injecting failure to force exercising of the consistency
codepaths in S3Guard.
> Requirements:
> - Integration tests that cause S3A to see the types of inconsistency we address with
> - These are deterministic integration tests.
> Unit tests are possible as well, if we were to stub out the S3Client.  That may be less
bang for the buck, though.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message