hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14553) Add (parallelized) integration tests to hadoop-azure
Date Thu, 06 Jul 2017 13:35:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076504#comment-16076504

Steve Loughran commented on HADOOP-14553:

-public class TestNativeAzureFileSystemContractMocked extends
+ * Mocked testing of FileSystemContractBaseTest.
+ * This isn't an IT, but making it so makes it a lot faster for now.
+ */
+public class ITestNativeAzureFileSystemContractMocked extends
bq.  why is it faster as ITest?

its  not that the test finishes fast, it's just as something slow, running it in parallel
meant the test run took less time.

I want to do another iteration of this and 

* rename Test* which requires credentails to being an ITest —but just list them in the sequential
* leave the other tests alone
* change the test profile in the POM to run the normal test profile without looking for an
auth-keys file

Goal: Jenkins/yetus to run the unit tests; move everything else to integration tests sooner
rather than later, and so allow for 1+ followup which parallelised the remaining tests, or
in the case of the big native test suite, split it up.

Regarding commonality between S3A test runner and the new stuff, yes, I did copy and past
S3ATestUtils in, which you would have noticed. Trouble is: I don't know what commonality we
really have right now. 

> Add (parallelized) integration tests to hadoop-azure
> ----------------------------------------------------
>                 Key: HADOOP-14553
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14553
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 2.9.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-14553-001.patch, HADOOP-14553-002.patch, HADOOP-14553-003.patch,
> The Azure tests are slow to run as they are serialized, as they are all called Test*
there's no clear differentiation from unit tests which Jenkins can run, and integration tests
which it can't.
> Move the azure tests {{Test*}} to integration tests {{ITest*}}, parallelize (which includes
having separate paths for every test suite). The code in hadoop-aws's POM &c show what
to do.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message