ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy Sorokin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-5302) Empty LOST partition may be used as OWNING after resetting lost partitions
Date Tue, 19 Dec 2017 14:49:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296899#comment-16296899
] 

Dmitriy Sorokin commented on IGNITE-5302:
-----------------------------------------

The last research results for this ticket (on the head of branch ignite-5267):

Without grid activation after start of nodes (last line in code block below)
{code}
        IgniteEx ignite1 = (IgniteEx)G.start(getConfiguration("test1"));
        IgniteEx ignite2 = (IgniteEx)G.start(getConfiguration("test2"));
        IgniteEx ignite3 = (IgniteEx)G.start(getConfiguration("test3"));
        IgniteEx ignite4 = (IgniteEx)G.start(getConfiguration("test4"));

        ignite1.active(true);
{code}
test fails with exception shown below:
{noformat}
class org.apache.ignite.IgniteException: Can not perform the operation because the cluster
is inactive. Note, that the cluster is considered inactive by default if Ignite Persistent
Store is used to let all the nodes join the cluster. To activate the cluster call Ignite.activate(true).

	at org.apache.ignite.internal.IgniteKernal.checkClusterState(IgniteKernal.java:3693)
	at org.apache.ignite.internal.IgniteKernal.cache(IgniteKernal.java:2713)
	at org.apache.ignite.internal.processors.cache.persistence.IgnitePdsCacheRebalancingAbstractTest.testPartitionLossAndRecover(IgnitePdsCacheRebalancingAbstractTest.java:336)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at junit.framework.TestCase.runTest(TestCase.java:176)
	at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:1995)
	at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
	at org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1910)
	at java.lang.Thread.run(Thread.java:745)
{noformat}

With grid activation test fails with assertion error:
{noformat}
java.lang.AssertionError
	at org.apache.ignite.internal.processors.cache.persistence.IgnitePdsCacheRebalancingAbstractTest.testPartitionLossAndRecover(IgnitePdsCacheRebalancingAbstractTest.java:350)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at junit.framework.TestCase.runTest(TestCase.java:176)
	at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:1995)
	at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
	at org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1910)
	at java.lang.Thread.run(Thread.java:745)
{noformat}

Assertion error shown above happens on that code line:
{code}
assert !ignite1.cache(cacheName).lostPartitions().isEmpty();
{code}

So, we need provide lost partitions in our test first, then we'll can fix the problem.

> Empty LOST partition may be used as OWNING after resetting lost partitions
> --------------------------------------------------------------------------
>
>                 Key: IGNITE-5302
>                 URL: https://issues.apache.org/jira/browse/IGNITE-5302
>             Project: Ignite
>          Issue Type: Bug
>            Reporter: Sergey Chugunov
>            Priority: Blocker
>              Labels: MakeTeamcityGreenAgain, Muted_test, test-fail
>             Fix For: 2.4
>
>
> h2. Notes
> Test *testPartitionLossAndRecover* reproducing the issue can be found in ignite-5267
branch with PDS functionality.
> h2. Steps to reproduce
> # Four nodes are started, some key is added to partitioned cache
> # Primary and backup nodes for the key are stopped, key's partition is declared LOST
on remaining nodes
> # Primary and backup nodes are started again, cache's lost partitions are reset
> # Key is requested from cache
> h2. Expected behavior
> Correct value is returned from primary for this partition
> h2. Actual behavior
> Request for value is sent to node where partition is empty (not to primary node), null
is returned
> h2. Latest findings
> # The main problem with the scenario is that request for key gets mapped not only to
P/B nodes with real value but also to the node where that partition existed only in LOST state
after P/B shutdown on step #2
> # It was found that on step #3 after primary and backup are joined partition counter
is increased for empty partition in LOST state which looks wrong



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message