ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Fedotov <ivanan...@gmail.com>
Subject Re: IgniteConfigVariationsAbstractTest subclasses do not run
Date Fri, 31 May 2019 12:37:36 GMT
Igniters,

I created corresponding tickets [1][2][3][4] in Jira and outlined
description of problems in a nutshell. In the context of the current ticket
(IGNITE-11708), I ignored them.

If some of us have a comprehension of the problem why tests are failed,
please let know here or in the tickets.

[1] https://issues.apache.org/jira/browse/IGNITE-11883
[2] https://issues.apache.org/jira/browse/IGNITE-11884
[3] https://issues.apache.org/jira/browse/IGNITE-11885
[4] https://issues.apache.org/jira/browse/IGNITE-11886



чт, 30 мая 2019 г. в 14:55, Ivan Fedotov <ivanan639@gmail.com>:

> Hi Igniters!
>
> I found the solution for how to run IgniteConfigVariationsAbstractTest. I
> removed garbage injection [1] and static variable [2].
> Now assigning of VariationsTestsConfig proceeds in constructor class which
> is created dynamically [3].
>
> But I faced with the problem that a certain amount of tests are failed. It
> seems that they failed because of the real reasons, not because
> of the test workflow. Despite those fact that a big amount of tests on TC
> became red, really failed tests are not so much. Derivatives tests appear
> from
> different configs.
>
> Could some of us take a look on failed tests? I am going to ignore them in
> the context of the current ticket [5] and create separate
> issues.
>
> [1]
> https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L102
> [2]
> https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L84
> [3]
> https://github.com/apache/ignite/pull/6434/files#diff-3da5dbb6f22e5bf3bf18734933f1bfc5R434
> [4]
> https://ci.ignite.apache.org/viewLog.html?buildId=3978807&buildTypeId=IgniteTests24Java8_RunAllNightly
> [5]https://issues.apache.org/jira/browse/IGNITE-11708
>
> вт, 14 мая 2019 г. в 16:31, Ivan Pavlukhina <vololo100@gmail.com>:
>
>> Ivan F.,
>>
>> Agree with referring tickets in @Ignore annotation.
>>
>> Currently I have no access to a computer. Will be able to look at updated
>> PR next week.
>>
>> Sent from my iPhone
>>
>> > On 14 May 2019, at 10:55, Ivan Fedotov <ivanan639@gmail.com> wrote:
>> >
>> > Ivan P.,
>> > I managed with IgniteConfigVariationsAbstractTest - now subclasses
>> enable
>> > [1].
>> > I ignored all the failed tests that were mentioned in our conversation.
>> If
>> > you don't mind, I can create appropriate tickets and mention them in
>> Ignore
>> > annotation.
>> >
>> > [1] https://github.com/apache/ignite/pull/6434/files
>> >
>> > ср, 1 мая 2019 г. в 15:29, Павлухин Иван <vololo100@gmail.com>:
>> >
>> >> Ivan F.,
>> >>
>> >> I think that it is better to enable IgniteConfigVariationsAbstractTest
>> >> subclasses first, so no new broken tests will appear. After that we
>> >> can fix IgniteConfigVariationsAbstractTest subclasses one by one in
>> >> separate ticket(s).
>> >>
>> >> вт, 30 апр. 2019 г. в 13:23, Ivan Fedotov <ivanan639@gmail.com>:
>> >>>
>> >>> Ivan R., Ivan P., thank you for the suggestions, I took a look at
>> them.
>> >>>
>> >>> According to checking CacheAtomicityMode on null in
>> >>> IgniteCacheConfigVariationsAbstractTest#atomicityMode - are you sure
>> >>> that checking should proceed on test level? May be it will be better
>> to
>> >>> make default cache value in case if cc.getAtomicityMode() == null
>> >>> in CacheConfiguration constructor [1]?
>> >>>
>> >>> Also let me suggest one minor change according cache specification in
>> >>> IgniteCacheReadThroughEvictionSelfTest. The same logic is used in
>> >>> GridCacheAbstractSelfTest#cacheConfiguration [2].
>> >>> I think we can excract this code block in a separate static methods
>> >>> (initCacheConfig, for example) in
>> IgniteCacheReadThroughEvictionSelfTest
>> >> and
>> >>> invoke it in IgniteCacheReadThroughEvictionSelfTest#variationConfig
>> and
>> >>> GridCacheAbstractSelfTest#cacheConfiguration methods.
>> >>>
>> >>> In addition, I want to draw your attention to
>> >>> IgniteContinuousQueryConfigVariationsSuite test runs [3].
>> >>> CacheContinuousQueryVariationsTest are generated dynamically.
>> >>> The first batch (12 tests) works fine, but on the next runs we got
>> NPE in
>> >>> IgniteCacheConfigVariationsAbstractTest#afterTest - default cache does
>> >> not
>> >>> exist and we can not
>> >>> invoke unwrap() on this [4].
>> >>> Seems that the problem is in
>> >>>
>> >>
>> IgniteCacheConfigVariationsAbstractTest#beforeTestsStarted/afterTestsStopped
>> >>> methods, cache is not properly created (or already existed cache is
>> >>> destroyed).
>> >>>
>> >>> By the way, should I resolve these issues in the context of
>> IGNITE-11708
>> >> or
>> >>> create a separate ticket(s)?
>> >>>
>> >>> [1]
>> >>>
>> >>
>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CacheConfiguration.java#L434
>> >>> [2]
>> >>>
>> >>
>> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheAbstractSelfTest.java#L235
>> >>> [3]
>> >>>
>> >>
>> https://ci.ignite.apache.org/viewLog.html?buildId=3701865&buildTypeId=IgniteTests24Java8_RunAllNightly
>> >>> [4]
>> >>>
>> >>
>> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteCacheConfigVariationsAbstractTest.java#L212
>> >>>
>> >>> пт, 26 апр. 2019 г. в 18:01, Ivan Rakov <ivan.glukos@gmail.com>:
>> >>>
>> >>>> Ivan P.,
>> >>>>
>> >>>> Good catch, thanks.
>> >>>>
>> >>>> I was wrong, test scenario is correct. The problem was in
>> >>>> atomicityMode() method - it could have returned null (which was
okay
>> >> for
>> >>>> config generation, but wasn't expected in the test code).
>> >>>> Please take a look at tx_out_test_fixed.patch (attached to
>> >>>> IGNITE-11708). To sum it up, both issues should be fixed now.
>> >>>>
>> >>>> Best Regards,
>> >>>> Ivan Rakov
>> >>>>
>> >>>>> On 26.04.2019 14:40, Павлухин Иван wrote:
>> >>>>> Ivan R.,
>> >>>>>
>> >>>>> As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx
>> does
>> >>>>> not expect lock/unlock events due to line:
>> >>>>> if (atomicityMode() == ATOMIC)
>> >>>>>     return lockEvtCnt.get() == 0;
>> >>>>>
>> >>>>> Could you please elaborate?
>> >>>>>
>> >>>>> пт, 26 апр. 2019 г. в 13:32, Ivan Rakov <ivan.glukos@gmail.com>:
>> >>>>>> Ivan,
>> >>>>>>
>> >>>>>> Seems like IgniteCacheReadThroughEvictionSelfTest is broken.
Test
>> >>>>>> scenario assumes that even after expiration entry will be
present
>> in
>> >>>>>> IgniteCache as per it will be loaded from CacheStore. However,
>> >>>>>> CacheStore is not specified in node config. I've added patch
that
>> >>>>>> enables cache store factory, please check IGNITE-11708 attachments.
>> >>>>>> Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx*
>> >> tests:
>> >>>>>> from my point of view, test scenarios make no sense. We
perform
>> >> get()
>> >>>>>> operation from ATOMIC caches and expect that entries will
be
>> >> locked. I
>> >>>>>> don't understand why we should lock entries on ATOMIC get,
>> >> therefore I
>> >>>>>> suppose to remove part of code where we listen and check
>> >>>>>> EVT_CACHE_OBJECT_LOCKED/UNLOCKED events.
>> >>>>>>
>> >>>>>> Best Regards,
>> >>>>>> Ivan Rakov
>> >>>>>>
>> >>>>>>> On 17.04.2019 22:05, Ivan Rakov wrote:
>> >>>>>>> Hi Ivan,
>> >>>>>>>
>> >>>>>>> I've checked your branch. Seems like these tests fail
due to real
>> >>>>>>> issue in functionality.
>> >>>>>>> I'll take a look.
>> >>>>>>>
>> >>>>>>> Best Regards,
>> >>>>>>> Ivan Rakov
>> >>>>>>>
>> >>>>>>>> On 17.04.2019 13:54, Ivan Fedotov wrote:
>> >>>>>>>> Hi, Igniters!
>> >>>>>>>>
>> >>>>>>>> During work on iep-30[1] I discovered that
>> >>>>>>>> IgniteConfigVariationsAbstractTest subclasses -
it is about
>> 15_000
>> >>>>>>>> tests[2]
>> >>>>>>>> - do not work.
>> >>>>>>>> You can check it just run one of the tests with
log output, for
>> >>>> example
>> >>>>>>>> ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1
>> >> [3].
>> >>>>>>>> There is no warning notification in the console.
The same
>> >> situation
>> >>>> with
>> >>>>>>>> other IgniteConfigVariationsAbstractTest subclasses
- tests run,
>> >> but
>> >>>>>>>> they
>> >>>>>>>> simply represent empty code.
>> >>>>>>>>
>> >>>>>>>> So, I created a ticket on such issue [4] and it
turned out that
>> >> the
>> >>>>>>>> problem
>> >>>>>>>> is with ruleChain in IgniteConfigVariationsAbstractTest
[5].
>> >>>>>>>> The rule that is responsible for running a test
statement does
>> not
>> >>>> start
>> >>>>>>>> indeed [6] under ruleChain runRule. I suggested
a solution - move
>> >>>>>>>> testsCfg
>> >>>>>>>> initialization to
>> >>>>>>>> IgniteConfigVariationsAbstractTest#beforeTestsStarted
method.
>> >> After
>> >>>> such
>> >>>>>>>> changes ruleChain becomes not necessary.
>> >>>>>>>>
>> >>>>>>>> But I faced another problem - multiple failures
on TeamCity [7].
>> >> From
>> >>>>>>>> logs,
>> >>>>>>>> it seems that failures are related to what tests
check, but not
>> >> JUnit
>> >>>>>>>> error.
>> >>>>>>>> I can not track TeamCity history on that fact were
tests failed
>> or
>> >>>>>>>> not on
>> >>>>>>>> the previous JUnit version - the oldest log is dated
the start of
>> >> the
>> >>>>>>>> March
>> >>>>>>>> when JUnit4
>> >>>>>>>> already was implemented (for example, this [8] test).
>> >>>>>>>> Moreover, there are not so much failed tests, but
because of
>> >> running
>> >>>>>>>> with
>> >>>>>>>> multiple configurations
>> >>>>>>>> (InterceptorCacheConfigVariationsFullApiTestSuite_0
>> >>>>>>>> ..._95) it turns out about 400 failed tests. TeamCity
results
>> also
>> >>>>>>>> confirm
>> >>>>>>>> that tests do not work in the master branch - duration
time is
>> >> less
>> >>>> than
>> >>>>>>>> 1ms. Now all tests are green and that is not surprising
- under
>> >> @Test
>> >>>>>>>> annotation, nothing happens.
>> >>>>>>>>
>> >>>>>>>> Could some of us confirm or disprove my guess that
tests are red
>> >>>>>>>> because of
>> >>>>>>>> its functionality, but not JUnit implementation?
>> >>>>>>>> And if it is true, how should I take such fact into
account in
>> >> this
>> >>>>>>>> ticket?
>> >>>>>>>>
>> >>>>>>>> [1]
>> >>>>>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5
>> >>>>>>>>
>> >>>>>>>> [2]
>> >>>>>>>>
>> >>>>
>> >>
>> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java
>> >>>>>>>>
>> >>>>>>>> [3]
>> >>>>>>>>
>> >>>>
>> >>
>> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434
>> >>>>>>>>
>> >>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-11708
>> >>>>>>>> [5]
>> >>>>>>>>
>> >>>>
>> >>
>> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62
>> >>>>>>>>
>> >>>>>>>> [6]
>> >>>>>>>>
>> >>>>
>> >>
>> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181
>> >>>>>>>>
>> >>>>>>>> [7]
>> >>>>>>>>
>> >>>>
>> >>
>> https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest
>> >>>>>>>>
>> >>>>>>>> [8]
>> >>>>>>>>
>> >>>>
>> >>
>> https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> Ivan Fedotov.
>> >>>
>> >>> ivanan639@gmail.com
>> >>
>> >>
>> >>
>> >> --
>> >> Best regards,
>> >> Ivan Pavlukhin
>> >>
>> >
>> >
>> > --
>> > Ivan Fedotov.
>> >
>> > ivanan639@gmail.com
>>
>
>
> --
> Ivan Fedotov.
>
> ivanan639@gmail.com
>


-- 
Ivan Fedotov.

ivanan639@gmail.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message