hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14553) Add (parallelized) integration tests to hadoop-azure
Date Thu, 06 Jul 2017 13:35:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076504#comment-16076504
] 

Steve Loughran commented on HADOOP-14553:
-----------------------------------------

{code}
-public class TestNativeAzureFileSystemContractMocked extends
+/**
+ * Mocked testing of FileSystemContractBaseTest.
+ * This isn't an IT, but making it so makes it a lot faster for now.
+ */
+public class ITestNativeAzureFileSystemContractMocked extends
{code}
bq.  why is it faster as ITest?

its  not that the test finishes fast, it's just as something slow, running it in parallel
meant the test run took less time.

I want to do another iteration of this and 

* rename Test* which requires credentails to being an ITest —but just list them in the sequential
section
* leave the other tests alone
* change the test profile in the POM to run the normal test profile without looking for an
auth-keys file

Goal: Jenkins/yetus to run the unit tests; move everything else to integration tests sooner
rather than later, and so allow for 1+ followup which parallelised the remaining tests, or
in the case of the big native test suite, split it up.


Regarding commonality between S3A test runner and the new stuff, yes, I did copy and past
S3ATestUtils in, which you would have noticed. Trouble is: I don't know what commonality we
really have right now. 

> Add (parallelized) integration tests to hadoop-azure
> ----------------------------------------------------
>
>                 Key: HADOOP-14553
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14553
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 2.9.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-14553-001.patch, HADOOP-14553-002.patch, HADOOP-14553-003.patch,
HADOOP-14553-004.patch
>
>
> The Azure tests are slow to run as they are serialized, as they are all called Test*
there's no clear differentiation from unit tests which Jenkins can run, and integration tests
which it can't.
> Move the azure tests {{Test*}} to integration tests {{ITest*}}, parallelize (which includes
having separate paths for every test suite). The code in hadoop-aws's POM &c show what
to do.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message