Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 1 Jun 2017 14:14:04 +0000 (UTC)
From: "Steve Loughran (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.13076545.1496325359000.341712.1496326444125@Atlassian.JIRA>
In-Reply-To: <JIRA.13076545.1496325359000@Atlassian.JIRA>
References: <JIRA.13076545.1496325359000@Atlassian.JIRA> <JIRA.13076545.1496325359718@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HADOOP-14476) make InconsistentAmazonS3Client
 usable in downstream tests
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 01 Jun 2017 14:14:10 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033040#comment-16033040 ] 

Steve Loughran commented on HADOOP-14476:
-----------------------------------------

+add test to verify that if delayed listing is on, a probe for a directory, that is, getFileStatus(), may fail because no children are found. This is something which has been observed in committers where the output==1 file and so the taskAttemptDir is not found

> make InconsistentAmazonS3Client usable in downstream tests
> ----------------------------------------------------------
>
>                 Key: HADOOP-14476
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14476
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3, test
>    Affects Versions: HADOOP-13345
>            Reporter: Steve Loughran
>
> It's important for downstream apps to be able to verify that s3guard works by making the AWS client inconsistent (so demonstrate problems), then turn s3guard on to verify that they go away. 
> This can be done by exposing the {{InconsistentAmazonS3Client}}
> # move the factory to the production source
> # make delay configurable for when you want a really long delay
> # have factory code log @ warn when a non-default factory is used.
> # mention in s3a testing.md
> I think we could look at the name of the option, {{fs.s3a.s3.client.factory.impl}} too. I'd like something which has "internal" in it, and without the duplication of s3a.s3


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org