Return-Path: <common-issues-return-111804-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id 0FF90200B2B
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Mon, 13 Jun 2016 18:41:23 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id 0ED1D160A1A; Mon, 13 Jun 2016 16:41:23 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id 5EA9F160A3C
	for <archive-asf-public@cust-asf.ponee.io>; Mon, 13 Jun 2016 18:41:22 +0200 (CEST)
Received: (qmail 15320 invoked by uid 500); 13 Jun 2016 16:41:21 -0000
Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:common-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:common-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:common-issues@hadoop.apache.org>
List-Id: <common-issues.hadoop.apache.org>
Delivered-To: mailing list common-issues@hadoop.apache.org
Received: (qmail 15282 invoked by uid 99); 13 Jun 2016 16:41:21 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Jun 2016 16:41:21 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 311062C1F5C
	for <common-issues@hadoop.apache.org>; Mon, 13 Jun 2016 16:41:21 +0000 (UTC)
Date: Mon, 13 Jun 2016 16:41:21 +0000 (UTC)
From: "Chris Nauroth (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.12974943.1464817163000.71602.1465836081196@Atlassian.JIRA>
In-Reply-To: <JIRA.12974943.1464817163000@Atlassian.JIRA>
References: <JIRA.12974943.1464817163000@Atlassian.JIRA> <JIRA.12974943.1464817163865@arcas>
Subject: [jira] [Commented] (HADOOP-13231) Isolate test path used by a few
 S3A tests for more reliable parallel execution.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
archived-at: Mon, 13 Jun 2016 16:41:23 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-13231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327714#comment-15327714 ] 

Chris Nauroth commented on HADOOP-13231:
----------------------------------------

bq. Should those named tests be .class rather than .java?

Using .java is correct, as per Maven's documentation on [Inclusions and Exclusions of Tests|http://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html].

Repeating a comment from HADOOP-13139:

{code}
mvn test -Dtest=TestS3A*  -Pparallel-tests -DtestsThreadCount=2
{code}

[~stevel@apache.org], when you pass a {{-Dtest=}} argument, you are overriding the {{<includes>}} list passed to Surefire during the build.  Passing {{TestS3A*}} matches all the S3A test suites, including the ones I listed in {{<excludes>}} to prevent parallel execution.  I just tested this, and it looks like if Surefire finds the same test matching both the whitelist and the blacklist, then the whitelist wins.  Therefore, tests that can't handle parallel execution end up running in the parallel phase.

Unfortunately, I think this means you can't reliably do both {{-Pparallel-tests}} and {{-Dtest=TestS3A*}} in the same {{mvn}} run.  I had flagged all multi-part upload tests to run outside the parallel phase because of the abort-existing-uploads-on-init logic, so I wouldn't expect them to work well in the parallel phase.

It would be nice to be able to run only S3A tests in parallel for more rapid iterations on patches that touch only S3A code.  I think we'd need to use some other naming convention on all non-parallel-safe test suite classes to support that, so that you could specify a wildcard to {{-Dtest=}} that wouldn't match those suites.

> Isolate test path used by a few S3A tests for more reliable parallel execution.
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-13231
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13231
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3, test
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>            Priority: Minor
>
> I have noticed a few more spots in S3A tests that do not make use of the isolated test directory path when running in parallel mode.  While I don't have any evidence that this is really causing problems for parallel test runs right now, it would still be good practice to clean these up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org