ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Fedotov <ivanan...@gmail.com>
Subject Re: IgniteConfigVariationsAbstractTest subclasses do not run
Date Thu, 30 May 2019 11:55:18 GMT
Hi Igniters!

I found the solution for how to run IgniteConfigVariationsAbstractTest. I
removed garbage injection [1] and static variable [2].
Now assigning of VariationsTestsConfig proceeds in constructor class which
is created dynamically [3].

But I faced with the problem that a certain amount of tests are failed. It
seems that they failed because of the real reasons, not because
of the test workflow. Despite those fact that a big amount of tests on TC
became red, really failed tests are not so much. Derivatives tests appear
from
different configs.

Could some of us take a look on failed tests? I am going to ignore them in
the context of the current ticket [5] and create separate
issues.

[1]
https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L102
[2]
https://github.com/apache/ignite/pull/6434/files#diff-cd3a07ce10f21d396c1eac0c328850e0L84
[3]
https://github.com/apache/ignite/pull/6434/files#diff-3da5dbb6f22e5bf3bf18734933f1bfc5R434
[4]
https://ci.ignite.apache.org/viewLog.html?buildId=3978807&buildTypeId=IgniteTests24Java8_RunAllNightly
[5]https://issues.apache.org/jira/browse/IGNITE-11708

вт, 14 мая 2019 г. в 16:31, Ivan Pavlukhina <vololo100@gmail.com>:

> Ivan F.,
>
> Agree with referring tickets in @Ignore annotation.
>
> Currently I have no access to a computer. Will be able to look at updated
> PR next week.
>
> Sent from my iPhone
>
> > On 14 May 2019, at 10:55, Ivan Fedotov <ivanan639@gmail.com> wrote:
> >
> > Ivan P.,
> > I managed with IgniteConfigVariationsAbstractTest - now subclasses enable
> > [1].
> > I ignored all the failed tests that were mentioned in our conversation.
> If
> > you don't mind, I can create appropriate tickets and mention them in
> Ignore
> > annotation.
> >
> > [1] https://github.com/apache/ignite/pull/6434/files
> >
> > ср, 1 мая 2019 г. в 15:29, Павлухин Иван <vololo100@gmail.com>:
> >
> >> Ivan F.,
> >>
> >> I think that it is better to enable IgniteConfigVariationsAbstractTest
> >> subclasses first, so no new broken tests will appear. After that we
> >> can fix IgniteConfigVariationsAbstractTest subclasses one by one in
> >> separate ticket(s).
> >>
> >> вт, 30 апр. 2019 г. в 13:23, Ivan Fedotov <ivanan639@gmail.com>:
> >>>
> >>> Ivan R., Ivan P., thank you for the suggestions, I took a look at them.
> >>>
> >>> According to checking CacheAtomicityMode on null in
> >>> IgniteCacheConfigVariationsAbstractTest#atomicityMode - are you sure
> >>> that checking should proceed on test level? May be it will be better to
> >>> make default cache value in case if cc.getAtomicityMode() == null
> >>> in CacheConfiguration constructor [1]?
> >>>
> >>> Also let me suggest one minor change according cache specification in
> >>> IgniteCacheReadThroughEvictionSelfTest. The same logic is used in
> >>> GridCacheAbstractSelfTest#cacheConfiguration [2].
> >>> I think we can excract this code block in a separate static methods
> >>> (initCacheConfig, for example) in
> IgniteCacheReadThroughEvictionSelfTest
> >> and
> >>> invoke it in IgniteCacheReadThroughEvictionSelfTest#variationConfig and
> >>> GridCacheAbstractSelfTest#cacheConfiguration methods.
> >>>
> >>> In addition, I want to draw your attention to
> >>> IgniteContinuousQueryConfigVariationsSuite test runs [3].
> >>> CacheContinuousQueryVariationsTest are generated dynamically.
> >>> The first batch (12 tests) works fine, but on the next runs we got NPE
> in
> >>> IgniteCacheConfigVariationsAbstractTest#afterTest - default cache does
> >> not
> >>> exist and we can not
> >>> invoke unwrap() on this [4].
> >>> Seems that the problem is in
> >>>
> >>
> IgniteCacheConfigVariationsAbstractTest#beforeTestsStarted/afterTestsStopped
> >>> methods, cache is not properly created (or already existed cache is
> >>> destroyed).
> >>>
> >>> By the way, should I resolve these issues in the context of
> IGNITE-11708
> >> or
> >>> create a separate ticket(s)?
> >>>
> >>> [1]
> >>>
> >>
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CacheConfiguration.java#L434
> >>> [2]
> >>>
> >>
> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheAbstractSelfTest.java#L235
> >>> [3]
> >>>
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=3701865&buildTypeId=IgniteTests24Java8_RunAllNightly
> >>> [4]
> >>>
> >>
> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteCacheConfigVariationsAbstractTest.java#L212
> >>>
> >>> пт, 26 апр. 2019 г. в 18:01, Ivan Rakov <ivan.glukos@gmail.com>:
> >>>
> >>>> Ivan P.,
> >>>>
> >>>> Good catch, thanks.
> >>>>
> >>>> I was wrong, test scenario is correct. The problem was in
> >>>> atomicityMode() method - it could have returned null (which was okay
> >> for
> >>>> config generation, but wasn't expected in the test code).
> >>>> Please take a look at tx_out_test_fixed.patch (attached to
> >>>> IGNITE-11708). To sum it up, both issues should be fixed now.
> >>>>
> >>>> Best Regards,
> >>>> Ivan Rakov
> >>>>
> >>>>> On 26.04.2019 14:40, Павлухин Иван wrote:
> >>>>> Ivan R.,
> >>>>>
> >>>>> As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx
does
> >>>>> not expect lock/unlock events due to line:
> >>>>> if (atomicityMode() == ATOMIC)
> >>>>>     return lockEvtCnt.get() == 0;
> >>>>>
> >>>>> Could you please elaborate?
> >>>>>
> >>>>> пт, 26 апр. 2019 г. в 13:32, Ivan Rakov <ivan.glukos@gmail.com>:
> >>>>>> Ivan,
> >>>>>>
> >>>>>> Seems like IgniteCacheReadThroughEvictionSelfTest is broken.
Test
> >>>>>> scenario assumes that even after expiration entry will be present
in
> >>>>>> IgniteCache as per it will be loaded from CacheStore. However,
> >>>>>> CacheStore is not specified in node config. I've added patch
that
> >>>>>> enables cache store factory, please check IGNITE-11708 attachments.
> >>>>>> Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx*
> >> tests:
> >>>>>> from my point of view, test scenarios make no sense. We perform
> >> get()
> >>>>>> operation from ATOMIC caches and expect that entries will be
> >> locked. I
> >>>>>> don't understand why we should lock entries on ATOMIC get,
> >> therefore I
> >>>>>> suppose to remove part of code where we listen and check
> >>>>>> EVT_CACHE_OBJECT_LOCKED/UNLOCKED events.
> >>>>>>
> >>>>>> Best Regards,
> >>>>>> Ivan Rakov
> >>>>>>
> >>>>>>> On 17.04.2019 22:05, Ivan Rakov wrote:
> >>>>>>> Hi Ivan,
> >>>>>>>
> >>>>>>> I've checked your branch. Seems like these tests fail due
to real
> >>>>>>> issue in functionality.
> >>>>>>> I'll take a look.
> >>>>>>>
> >>>>>>> Best Regards,
> >>>>>>> Ivan Rakov
> >>>>>>>
> >>>>>>>> On 17.04.2019 13:54, Ivan Fedotov wrote:
> >>>>>>>> Hi, Igniters!
> >>>>>>>>
> >>>>>>>> During work on iep-30[1] I discovered that
> >>>>>>>> IgniteConfigVariationsAbstractTest subclasses - it is
about 15_000
> >>>>>>>> tests[2]
> >>>>>>>> - do not work.
> >>>>>>>> You can check it just run one of the tests with log
output, for
> >>>> example
> >>>>>>>> ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1
> >> [3].
> >>>>>>>> There is no warning notification in the console. The
same
> >> situation
> >>>> with
> >>>>>>>> other IgniteConfigVariationsAbstractTest subclasses
- tests run,
> >> but
> >>>>>>>> they
> >>>>>>>> simply represent empty code.
> >>>>>>>>
> >>>>>>>> So, I created a ticket on such issue [4] and it turned
out that
> >> the
> >>>>>>>> problem
> >>>>>>>> is with ruleChain in IgniteConfigVariationsAbstractTest
[5].
> >>>>>>>> The rule that is responsible for running a test statement
does not
> >>>> start
> >>>>>>>> indeed [6] under ruleChain runRule. I suggested a solution
- move
> >>>>>>>> testsCfg
> >>>>>>>> initialization to
> >>>>>>>> IgniteConfigVariationsAbstractTest#beforeTestsStarted
method.
> >> After
> >>>> such
> >>>>>>>> changes ruleChain becomes not necessary.
> >>>>>>>>
> >>>>>>>> But I faced another problem - multiple failures on TeamCity
[7].
> >> From
> >>>>>>>> logs,
> >>>>>>>> it seems that failures are related to what tests check,
but not
> >> JUnit
> >>>>>>>> error.
> >>>>>>>> I can not track TeamCity history on that fact were tests
failed or
> >>>>>>>> not on
> >>>>>>>> the previous JUnit version - the oldest log is dated
the start of
> >> the
> >>>>>>>> March
> >>>>>>>> when JUnit4
> >>>>>>>> already was implemented (for example, this [8] test).
> >>>>>>>> Moreover, there are not so much failed tests, but because
of
> >> running
> >>>>>>>> with
> >>>>>>>> multiple configurations
> >>>>>>>> (InterceptorCacheConfigVariationsFullApiTestSuite_0
> >>>>>>>> ..._95) it turns out about 400 failed tests. TeamCity
results also
> >>>>>>>> confirm
> >>>>>>>> that tests do not work in the master branch - duration
time is
> >> less
> >>>> than
> >>>>>>>> 1ms. Now all tests are green and that is not surprising
- under
> >> @Test
> >>>>>>>> annotation, nothing happens.
> >>>>>>>>
> >>>>>>>> Could some of us confirm or disprove my guess that tests
are red
> >>>>>>>> because of
> >>>>>>>> its functionality, but not JUnit implementation?
> >>>>>>>> And if it is true, how should I take such fact into
account in
> >> this
> >>>>>>>> ticket?
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5
> >>>>>>>>
> >>>>>>>> [2]
> >>>>>>>>
> >>>>
> >>
> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java
> >>>>>>>>
> >>>>>>>> [3]
> >>>>>>>>
> >>>>
> >>
> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/test/ConfigVariationsTestSuiteBuilderTest.java#L434
> >>>>>>>>
> >>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-11708
> >>>>>>>> [5]
> >>>>>>>>
> >>>>
> >>
> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62
> >>>>>>>>
> >>>>>>>> [6]
> >>>>>>>>
> >>>>
> >>
> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java#L181
> >>>>>>>>
> >>>>>>>> [7]
> >>>>>>>>
> >>>>
> >>
> https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest
> >>>>>>>>
> >>>>>>>> [8]
> >>>>>>>>
> >>>>
> >>
> https://ci.ignite.apache.org/project.html?tab=testDetails&projectId=IgniteTests24Java8&testNameId=-9037806478172035481&page=8
> >>>>>>>>
> >>>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Ivan Fedotov.
> >>>
> >>> ivanan639@gmail.com
> >>
> >>
> >>
> >> --
> >> Best regards,
> >> Ivan Pavlukhin
> >>
> >
> >
> > --
> > Ivan Fedotov.
> >
> > ivanan639@gmail.com
>


-- 
Ivan Fedotov.

ivanan639@gmail.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message