geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anthony Baker <aba...@pivotal.io>
Subject Re: PutCommandWithJsonTest and DeployWithLargeJarTest
Date Tue, 21 Aug 2018 18:32:59 GMT
By design, acceptanceTest should be run with defaults (including ports).  So if they aren’t
run in containers they can’t be run in parallel.

Anthony


> On Aug 21, 2018, at 11:15 AM, Kirk Lund <klund@apache.org> wrote:
> 
> Oops... I mean "orphaned processes"
> 
> On Tue, Aug 21, 2018 at 11:13 AM, Kirk Lund <klund@apache.org> wrote:
> 
>> So, we need to disable running them in parallel and improve the tearDown
>> to make sure these tests don't leave an orphaned tests behind.
>> 
>> On Tue, Aug 21, 2018 at 11:11 AM, Sai Boorlagadda <
>> sai.boorlagadda@gmail.com> wrote:
>> 
>>> jdbc-connector acceptance tests need docker-in-docker (also
>>> docker-compose)
>>> to spin up mysql and postgres.
>>> 
>>> On Tue, Aug 21, 2018 at 11:04 AM Dan Smith <dsmith@pivotal.io> wrote:
>>> 
>>>> Actually, it looks like the problem is that we are *not* using docker
>>>> containers for the acceptance tests. Check this out, in
>>>> gradle/docker.gradle. Since acceptance tests use the default port, this
>>>> means the test are guaranteed to be flaky, especially since we are
>>> running
>>>> them in parallel:
>>>> 
>>>> //    ACCEPTANCE TEST NEEDS DOCKER-COMPOSE TO WORK WITHIN DOCKER FIRST
>>>> //    acceptanceTest.configure(dockerConfig)
>>>> 
>>>> I'm not sure what changed that is causing the tests to fail more often
>>> now,
>>>> but maybe a test ordering change?
>>>> 
>>>> -Dan
>>>> 
>>>> 
>>>> 
>>>> On Tue, Aug 21, 2018 at 10:52 AM, Kenneth Howe <khowe@pivotal.io>
>>> wrote:
>>>> 
>>>>> 
>>>>> 
>>>>>> On Aug 21, 2018, at 10:44 AM, Kirk Lund <klund@apache.org>
wrote:
>>>>>> 
>>>>>> GEODE-5590 would seem to imply that GfshRule does not have an
>>> adequate
>>>>> safe
>>>>>> guard? If it spawns a server process which binds to the default
>>> server
>>>>> port
>>>>>> and that process persists after the test then we need better
>>> tearDown.
>>>>>> 
>>>>> Yes, that does appear to be the case. The current failures are
>>> apparently
>>>>> due to incomplete
>>>>> teardown between tests within a test class.
>>>>> 
>>>>> I am attempting  to reproduce the failures on a consistent basis for
>>>>> debugging the problem.
>>>>> 
>>>>> 
>>>>>> Actually I thought we were using Docker to run each AcceptanceTest
>>> in
>>>>>> isolation. Then when the test finishes the Docker instances goes
>>> away.
>>>>> Did
>>>>>> we stop using Docker for these?
>>>>>> 
>>>>>> On Tue, Aug 21, 2018 at 10:25 AM, Sai Boorlagadda <
>>>>> sai.boorlagadda@gmail.com
>>>>>>> wrote:
>>>>>> 
>>>>>>> DeployWithLargeJarTest & PutCommandWithJsonTest are flaky
on
>>> Develop.
>>>>>>> 
>>>>>>> DeployWithLargeJarTest -
>>>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/
>>>>>>> develop/jobs/AcceptanceTest/builds/335
>>>>>>> PutCommandWithJsonTest -
>>>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/
>>>>>>> develop/jobs/AcceptanceTest/builds/334
>>>>>>> 
>>>>>>> On Tue, Aug 21, 2018 at 10:18 AM Sai Boorlagadda <
>>>>>>> sai.boorlagadda@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> The metrics job themselves will be green (as they complete
to
>>>> success)
>>>>>>> but
>>>>>>>> you can expand the get_metrics task output and see that build#20
>>>>> started
>>>>>>>> reporting these failures, so probably these are due to recent
>>> changes
>>>>> on
>>>>>>>> develop. I believe these metrics are from develop CI test
runs.
>>>>>>>> 
>>>>>>>> On Tue, Aug 21, 2018 at 10:15 AM Kirk Lund <klund@apache.org>
>>> wrote:
>>>>>>>> 
>>>>>>>>> Those metrics show AcceptanceTests consistently GREEN.
Do these
>>>>> metrics
>>>>>>>>> include test failures from pull request precheckin runs
like
>>> mine?
>>>> Or
>>>>>>> does
>>>>>>>>> it just cover CI test runs?
>>>>>>>>> 
>>>>>>>>> On Tue, Aug 21, 2018 at 10:09 AM, Sai Boorlagadda <
>>>>>>>>> sai.boorlagadda@gmail.com
>>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Metrics show these started failing recently.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/
>>>>>>> metrics/jobs/
>>>>>>>>>> GeodeAcceptanceTestMetrics/builds/20
>>>>>>>>>> 
>>>>>>>>>> On Tue, Aug 21, 2018 at 10:07 AM Kirk Lund <klund@apache.org>
>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Are PutCommandWithJsonTest and DeployWithLargeJarTest
known to
>>> be
>>>>>>>>> flaky?
>>>>>>>>>>> 
>>>>>>>>>>> My latest pull request failed with these two
failures and all I
>>>> did
>>>>>>>>> was
>>>>>>>>>>> extract LocalRegion.validateRegionName and improve
unit
>>> testing of
>>>>>>>>>>> RegionNameValidation. No other tests failed for
me.
>>>>>>>>>>> 
>>>>>>>>>>>> Task :geode-assembly:acceptanceTest
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:619
>>>>> 
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:620
>>>>> 
>>>>>>>>>>> 
>>>>>>>>> org.apache.geode.management.internal.cli.commands.
>>>>>>> PutCommandWithJsonTest
>>>>>>>>>>>> putWithJsonString FAILED
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:621
>>>>> 
>>>>>>>>>>>   org.junit.ComparisonFailure: expected:<[0]>
but was:<[1]>
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:622
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>>>>>>>>>> Method)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:623
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(
>>>>>>>>>> NativeConstructorAccessorImpl.java:62)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:624
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
>>>>>>>>>> DelegatingConstructorAccessorImpl.java:45)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:625
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.
>>>>>>>>>> awaitIfNecessary(GfshScript.java:117)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:626
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshRule.execute(
>>>>>>>>>> GfshRule.java:135)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:627
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.execute(
>>>>>>>>>> GfshScript.java:106)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:628
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> org.apache.geode.management.internal.cli.commands.
>>>>>>>>>> PutCommandWithJsonTest.putWithJsonString(
>>>>> PutCommandWithJsonTest.java:
>>>>>>> 55)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:629
>>>>> 
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:630
>>>>> 
>>>>>>>>>>> 
>>>>>>>>> org.apache.geode.management.internal.cli.commands.
>>>>>>> DeployWithLargeJarTest
>>>>>>>>>>>> deployLargeSetOfJars FAILED
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:631
>>>>> 
>>>>>>>>>>>   org.junit.ComparisonFailure: expected:<[0]>
but was:<[1]>
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:632
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>>>>>>>>>> Method)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:633
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(
>>>>>>>>>> NativeConstructorAccessorImpl.java:62)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:634
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
>>>>>>>>>> DelegatingConstructorAccessorImpl.java:45)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:635
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.
>>>>>>>>>> awaitIfNecessary(GfshScript.java:117)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:636
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshRule.execute(
>>>>>>>>>> GfshRule.java:135)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:637
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.execute(
>>>>>>>>>> GfshScript.java:106)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:638
>>>>> 
>>>>>>>>>>>       at
>>>>>>>>>>> org.apache.geode.management.internal.cli.commands.
>>>>>>>>>> DeployWithLargeJarTest.deployLargeSetOfJars(
>>>>>>> DeployWithLargeJarTest.java:
>>>>>>>>>> 41)
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:639
>>>>> 
>>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#
>>> L5b60bc1a:640
>>>>> 
>>>>>>>>>>>> Task :geode-assembly:acceptanceTest FAILED
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>> 
>> 


Mime
View raw message