hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11984) Enable parallel JUnit tests in pre-commit.
Date Tue, 19 May 2015 00:03:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549540#comment-14549540

Chris Nauroth commented on HADOOP-11984:

Some of the tests related to ZooKeeper are hitting bind exceptions.  Our {{ClientBaseWithFixes}}
is a fork of a ZooKeeper class, which calls ZooKeeper's {{PortAssignment}}.  That ZooKeeper
class is not capable of guaranteeing uniqueness of ports across multiple concurrent processes.
 I recently fixed this in ZOOKEEPER-2183 as part of speeding up their CI.  I might need to
explore some kind of partial port of those changes over here into Hadoop.

I can't repro the {{OutOfMemoryError}} problems after setting {{reuseForks}} to false.  I
don't know if I'm just overloading these hosts by running multiple processes with a maximum
1 GB heap.  I don't have much visibility into these hosts.  If anyone else out there knows
how to find host-level details, please let me know.  Otherwise, I'll probably need to coordinate
with infra.

> Enable parallel JUnit tests in pre-commit.
> ------------------------------------------
>                 Key: HADOOP-11984
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11984
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HADOOP-11984.001.patch, HADOOP-11984.002.patch, HADOOP-11984.003.patch,
HADOOP-11984.004.patch, HADOOP-11984.005.patch
> HADOOP-9287 and related issues implemented the parallel-tests Maven profile for running
JUnit tests in multiple concurrent processes.  This issue proposes to activate that profile
during pre-commit to speed up execution.

This message was sent by Atlassian JIRA

View raw message