hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14553) Add (parallelized) integration tests to hadoop-azure
Date Wed, 09 Aug 2017 15:23:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120068#comment-16120068
] 

Steve Loughran commented on HADOOP-14553:
-----------------------------------------

Test results, without scale tests, 13 min. Still slow, but better than before
{code}
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractRename
Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractMkdir
Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractCreate
Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractSeek
Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractGetFileStatus
Running org.apache.hadoop.fs.azure.integration.ITestAzureHugeFiles
Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractDistCp
Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractAppend
Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractOpen
Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractDelete
Tests run: 6, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 6.017 sec - in org.apache.hadoop.fs.azure.integration.ITestAzureHugeFiles
Running org.apache.hadoop.fs.azure.ITestAzureConcurrentOutOfBandIo
Tests run: 4, Failures: 0, Errors: 0, Skipped: 4, Time elapsed: 12.752 sec - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractDistCp
Running org.apache.hadoop.fs.azure.ITestAzureConcurrentOutOfBandIoWithSecureMode
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.579 sec - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractOpen
Running org.apache.hadoop.fs.azure.ITestAzureFileSystemErrorConditions
Tests run: 7, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 23.806 sec - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractAppend
Running org.apache.hadoop.fs.azure.ITestBlobTypeSpeedDifference
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.035 sec - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractDelete
Running org.apache.hadoop.fs.azure.ITestContainerChecks
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.99 sec - in org.apache.hadoop.fs.azure.ITestAzureConcurrentOutOfBandIo
Running org.apache.hadoop.fs.azure.ITestFileSystemOperationExceptionHandling
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.834 sec - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractRename
Running org.apache.hadoop.fs.azure.ITestFileSystemOperationExceptionMessage
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.655 sec - in org.apache.hadoop.fs.azure.ITestFileSystemOperationExceptionMessage
Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemAppend
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.711 sec - in org.apache.hadoop.fs.azure.ITestContainerChecks
Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemAtomicRenameDirList
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.189 sec - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemAtomicRenameDirList
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.323 sec - in org.apache.hadoop.fs.azure.ITestBlobTypeSpeedDifference
Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemClientLogging
Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractEmulator
Tests run: 43, Failures: 0, Errors: 0, Skipped: 43, Time elapsed: 0.75 sec - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractEmulator
Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractLive
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.137 sec - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemClientLogging
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 29.386 sec - in org.apache.hadoop.fs.azure.ITestAzureConcurrentOutOfBandIoWithSecureMode
Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractPageBlobLive
Running org.apache.hadoop.fs.azure.ITestNativeAzureFSAuthorizationCaching
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.955 sec - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractCreate
Running org.apache.hadoop.fs.azure.ITestReadAndSeekPageBlobAfterWrite
Tests run: 5, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 2.299 sec - in org.apache.hadoop.fs.azure.ITestReadAndSeekPageBlobAfterWrite
Running org.apache.hadoop.fs.azure.ITestWasbUriAndConfiguration
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.526 sec - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractMkdir
Running org.apache.hadoop.fs.azure.metrics.ITestAzureFileSystemInstrumentation
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.83 sec - in org.apache.hadoop.fs.azure.ITestAzureFileSystemErrorConditions
Tests run: 28, Failures: 0, Errors: 0, Skipped: 28, Time elapsed: 6.39 sec - in org.apache.hadoop.fs.azure.ITestNativeAzureFSAuthorizationCaching
Tests run: 19, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 10.93 sec - in org.apache.hadoop.fs.azure.ITestWasbUriAndConfiguration
Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 66.589 sec - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractSeek
Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 90.738 sec - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractGetFileStatus
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.239 sec - in org.apache.hadoop.fs.azure.ITestFileSystemOperationExceptionHandling
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 106.703 sec - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemAppend
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 98.492 sec - in org.apache.hadoop.fs.azure.metrics.ITestAzureFileSystemInstrumentation
Tests run: 43, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 112.222 sec - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractLive
Tests run: 43, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 113.314 sec - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractPageBlobLive

Results :

Tests run: 319, Failures: 0, Errors: 0, Skipped: 98

[INFO] 
[INFO] --- maven-failsafe-plugin:2.17:integration-test (sequential-integration-tests) @ hadoop-azure
---
[INFO] Failsafe report directory: /Users/stevel/Projects/hadoop-trunk/hadoop-tools/hadoop-azure/target/failsafe-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.hadoop.fs.azure.ITestBlobDataValidation
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.57 sec - in org.apache.hadoop.fs.azure.ITestBlobDataValidation
Running org.apache.hadoop.fs.azure.ITestBlockBlobInputStream
Tests run: 21, Failures: 0, Errors: 0, Skipped: 21, Time elapsed: 1.016 sec - in org.apache.hadoop.fs.azure.ITestBlockBlobInputStream
Running org.apache.hadoop.fs.azure.ITestFileSystemOperationsExceptionHandlingMultiThreaded
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.971 sec - in org.apache.hadoop.fs.azure.ITestFileSystemOperationsExceptionHandlingMultiThreaded
Running org.apache.hadoop.fs.azure.ITestFileSystemOperationsWithThreads
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 230.704 sec - in org.apache.hadoop.fs.azure.ITestFileSystemOperationsWithThreads
Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemAuthorizationWithOwner
Tests run: 27, Failures: 0, Errors: 0, Skipped: 27, Time elapsed: 3.698 sec - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemAuthorizationWithOwner
Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemConcurrencyLive
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.255 sec - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemConcurrencyLive
Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemLive
Tests run: 51, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 271.658 sec - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemLive
Running org.apache.hadoop.fs.azure.ITestOutOfBandAzureBlobOperationsLive
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.048 sec - in org.apache.hadoop.fs.azure.ITestOutOfBandAzureBlobOperationsLive
Running org.apache.hadoop.fs.azure.ITestWasbRemoteCallHelper
Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed: 1.371 sec - in org.apache.hadoop.fs.azure.ITestWasbRemoteCallHelper

Results :

Tests run: 154, Failures: 0, Errors: 0, Skipped: 58

[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (depcheck) @ hadoop-azure ---
[INFO] 
[INFO] --- maven-failsafe-plugin:2.17:verify (default-integration-test) @ hadoop-azure ---
[INFO] Failsafe report directory: /Users/stevel/Projects/hadoop-trunk/hadoop-tools/hadoop-azure/target/failsafe-reports
[INFO] 
[INFO] --- maven-failsafe-plugin:2.17:verify (sequential-integration-tests) @ hadoop-azure
---
[INFO] Failsafe report directory: /Users/stevel/Projects/hadoop-trunk/hadoop-tools/hadoop-azure/target/failsafe-reports
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:03 min
[INFO] Finished at: 2017-08-09T16:12:31+01:00
[INFO] Final Memory: 34M/817M
[INFO] ------------------------------------------------------------------------
{code}

cleanup found 43 containers left afterwards, but I'd interrupted some previous test runs which
inevitably left some undeleted.



> Add (parallelized) integration tests to hadoop-azure
> ----------------------------------------------------
>
>                 Key: HADOOP-14553
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14553
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 2.9.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-14553-001.patch, HADOOP-14553-002.patch, HADOOP-14553-003.patch,
HADOOP-14553-004.patch, HADOOP-14553-005.patch, HADOOP-14553-006.patch, HADOOP-14553-007.patch,
HADOOP-14553-008.patch, HADOOP-14553-009.patch, HADOOP-14553-010.patch
>
>
> The Azure tests are slow to run as they are serialized, as they are all called Test*
there's no clear differentiation from unit tests which Jenkins can run, and integration tests
which it can't.
> Move the azure tests {{Test*}} to integration tests {{ITest*}}, parallelize (which includes
having separate paths for every test suite). The code in hadoop-aws's POM &c show what
to do.
> *UPDATE August 4, 2017*:  Adding a list of requirements to clarify the acceptance criteria
for this JIRA:
> # Parallelize test execution
> # Define test groups: i) UnitTests - self-contained, executed by Jenkins, ii) IntegrationTests
- requires Azure Storage account, executed by engineers prior to check-in, and if needed,
iii)	ScaleTests – long running performance and scalability tests.
> # Define configuration profiles to run tests with different settings.  Allows an engineer
to run “IntegrationTests” with fs.azure.secure.mode = true and false.  Need to review
settings to see what else would benefit.
> # Maven commands to run b) and c).  Turns out it is not easy to do with Maven, so we
might have to run it multiple times to run with different configuration settings.
> # Document how to add and run tests and the process for contributing to Apache Hadoop.
 Steve shared an example at https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md

> # UnitTests should run in under 2 minutes and IntegrationTests should run in under 15
minutes, even on slower network connections.  (These are rough goals)
> # Ensure test data (containers/blobs/etc) is deleted.  Exceptions for large persistent
content used repeatedly to expedite test execution. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message