hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-15003) Merge S3A committers into trunk: Yetus patch checker
Date Tue, 07 Nov 2017 17:30:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-15003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Steve Loughran updated HADOOP-15003:
    Attachment: HADOOP-13786-046.patch

Patch 046, tune tests + two spaces of indentation in the production code (checkstyle)

huge file IO
*{{ITestHugeMagicCommits}} skips {{test_040_PositionedReadHugeFile}}.
* All subclasses generate unique pathname for test, even under forked VM path
* added final "test case" which prints the StorageStatus for S3A (as this is shared across
instances, it gives the aggregate across the entire VM)

Mock IO
* override the S3A {{operationRetried()}} method. This had been triggering NPEs which were
being (correctly) swallowed in the retry logic...but they shouldn't have been raised at all.
* Use the word "mock" in all mock exceptions raised, to make clear they're mock exceptions

* cut ref to obsolete JetS3t test from pom.xml
* indent two spaces of {{FileOutputFormat}}

Aaron: I'm done here; there's nothing left to do in this patch. Code done, tests done, docs
done, downstream integration done. It's ready to go.
I don't think a separate branch will add much. Putting it into trunk and letting other people
play with it and its features (especially retry) is the way to finish it off. I don't think
I'll break anything, but a merge to trunk is the only way we will find this out. Also: makes
it easier for anyone to build & test the spark cloud-integration module, as they don't
need to build hadoop 3.1-snapshot. Again: gets broader use

> Merge S3A committers into trunk: Yetus patch checker
> ----------------------------------------------------
>                 Key: HADOOP-15003
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15003
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-13786-041.patch, HADOOP-13786-042.patch, HADOOP-13786-043.patch,
HADOOP-13786-044.patch, HADOOP-13786-045.patch, HADOOP-13786-046.patch
> This is a Yetus only JIRA created to have Yetus review the HADOOP-13786/HADOOP-14971
patch as a .patch file, as the review PR [https://github.com/apache/hadoop/pull/282] is stopping
this happening in HADOOP-14971.
> Reviews should go into the PR/other task

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message