hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14553) Add (parallelized) integration tests to hadoop-azure
Date Tue, 11 Jul 2017 21:08:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Steve Loughran updated HADOOP-14553:
    Attachment: HADOOP-14553-008.patch

Patch 008
* rebase to trunk & fix conflics
* move new test to an ITest

Its a scale one, only running if -Dscale is set. 

New test is failing 
Running org.apache.hadoop.fs.azure.ITestBlockBlobInputStream
Tests run: 19, Failures: 0, Errors: 17, Skipped: 0, Time elapsed: 8.839 sec <<< FAILURE!
- in org.apache.hadoop.fs.azure.ITestBlockBlobInputStream
test_0301_MarkSupportedV1(org.apache.hadoop.fs.azure.ITestBlockBlobInputStream)  Time elapsed:
0.425 sec  <<< ERROR!
java.io.FileNotFoundException: huge file not created: not found wasb://contract@contender.blob.core.windows.net/user/stevel/TestBlockBlobInputStream.txt
in wasb://contract@contender.blob.core.windows.net/user/stevel
	at org.apache.hadoop.fs.contract.ContractTestUtils.assertPathExists(ContractTestUtils.java:764)
	at org.apache.hadoop.fs.azure.ITestBlockBlobInputStream.assumeHugeFileExists(ITestBlockBlobInputStream.java:181)
	at org.apache.hadoop.fs.azure.ITestBlockBlobInputStream.validateMarkSupported(ITestBlockBlobInputStream.java:222)
	at org.apache.hadoop.fs.azure.ITestBlockBlobInputStream.test_0301_MarkSupportedV1(ITestBlockBlobInputStream.java:209)

I know why: teardown deletes the test path...I'll look at that, but first plan to merge all
these tests into the (new) huge file test {{ITestAzureHugeFiles}} so they can share the same
generated file. I'd chosen the numbering of the test methods for this already

> Add (parallelized) integration tests to hadoop-azure
> ----------------------------------------------------
>                 Key: HADOOP-14553
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14553
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 2.9.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-14553-001.patch, HADOOP-14553-002.patch, HADOOP-14553-003.patch,
HADOOP-14553-004.patch, HADOOP-14553-005.patch, HADOOP-14553-006.patch, HADOOP-14553-007.patch,
> The Azure tests are slow to run as they are serialized, as they are all called Test*
there's no clear differentiation from unit tests which Jenkins can run, and integration tests
which it can't.
> Move the azure tests {{Test*}} to integration tests {{ITest*}}, parallelize (which includes
having separate paths for every test suite). The code in hadoop-aws's POM &c show what
to do.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message