accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ACCUMULO-4124) integration tests that use map-reduce run under the yarn instance when using IntegrationTestMapReduce
Date Tue, 02 Feb 2016 17:38:39 GMT

     [ https://issues.apache.org/jira/browse/ACCUMULO-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eric Newton updated ACCUMULO-4124:
----------------------------------
    Description: 
While attempting to run the Integration Tests under a map-reduce job using the fat mrit jar,
I found that all the InputFormat and OutputFormat tests were failing. Checking on the cluster,
I saw that the ITs were kicking off new applications within the yarn cluster.

There are several problems with this:

1) the cluster is busy running ITs, so it doesn't have resources to be the M/R infrastructure
for a test
2) zookeeper is consistently started on localhost, and the sub-map jobs cannot reach the zookeeper
on the host that started it
3) files prepared by the test do not exist in order for the test to run because it is running
on another host

This contributes to 15 of the 192 tests throwing unexpected exceptions. That is, they do not
run to completion, either with test success, or test failure.  They also create a long tail
of timeout failures.


  was:
While attempting to run the Integration Tests under a map-reduce job using the fat mrit jar,
I found that all the InputFormat and OutputFormat tests were failing. Checking on the cluster,
I saw that the ITs were kicking off new applications with in the yarn cluster.

There are several problems with this:

1) the cluster is busy running ITs, so it doesn't have resources to be the M/R infrastructure
for a test
2) zookeeper is consistently started on localhost, and the sub-map jobs cannot reach the zookeeper
on the host that started it
3) files prepared by the test do not exist in order for the test to run because it is running
on another host

This contributes to 15 of the 192 tests to throw unexpected exceptions. That is, they do not
run to completion, either with test success, or test failure.  They also create a long tail
of timeout failures.



> integration tests that use map-reduce run under the yarn instance when using IntegrationTestMapReduce
> -----------------------------------------------------------------------------------------------------
>
>                 Key: ACCUMULO-4124
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4124
>             Project: Accumulo
>          Issue Type: Bug
>          Components: test
>         Environment: running integration tests as a map-reduce job on a small cluster
>            Reporter: Eric Newton
>            Assignee: Eric Newton
>             Fix For: 1.8.0
>
>
> While attempting to run the Integration Tests under a map-reduce job using the fat mrit
jar, I found that all the InputFormat and OutputFormat tests were failing. Checking on the
cluster, I saw that the ITs were kicking off new applications within the yarn cluster.
> There are several problems with this:
> 1) the cluster is busy running ITs, so it doesn't have resources to be the M/R infrastructure
for a test
> 2) zookeeper is consistently started on localhost, and the sub-map jobs cannot reach
the zookeeper on the host that started it
> 3) files prepared by the test do not exist in order for the test to run because it is
running on another host
> This contributes to 15 of the 192 tests throwing unexpected exceptions. That is, they
do not run to completion, either with test success, or test failure.  They also create a long
tail of timeout failures.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message