ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ksenia Rybakova (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (IGNITE-3474) Grid hung during load test with warning "Failed to wait for partition map exchange"
Date Fri, 18 Nov 2016 14:52:59 GMT

     [ https://issues.apache.org/jira/browse/IGNITE-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ksenia Rybakova updated IGNITE-3474:
------------------------------------
    Description: 
During load test grid hung:
{noformat}
[09:22:17,688][WARN ][exchange-worker-#137%null%][GridCachePartitionExchangeManager] Failed
to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=97, minorTopVer=0],
node=0fe0c0f2-65e1-4ad5-a8c9-92db997fcb7d]. Dumpin
g pending objects that might be the cause:
...
{noformat}
See attached file with more info from logs.

Notes:
The issue was also reproduced with IGNITE-3239 fixed. So, IGNITE-3239 doesn't seem to be a
root cause. 

Configs:
1)
 - Benchmark name: IgniteCacheRandomOperationBenchmark 
 - Operations: put, put_all, get, get_all, invoke, invoke_all, remove, remove_all, put_if_absent,
replace
- Heap: 4Gb for servers, 1Gb for clients
- 80 clients, 20 servers, 8 hosts
- Number of caches: 100
- Preload amount 250K
- Key range 500K
- Types of caches (atomicity mode): different (atomic, transactional)
- Types of caches (tiered storage mode): different (onheap without eviction, onheap with eviction,
offheap_tired, offheap_values)
- Types of caches (indexing): different (with and without indexes)
- Types of caches (cache mode): different (partitioned, replicated)
- Backups count: 1
- Duration: 1h

2) 
- Benchmark name: IgniteCacheRandomOperationBenchmark 
- Operations: scan_query, sql_query, continuous_query
- Heap: 4Gb for servers, 1Gb for clients
- 20 clients, 80 servers, 8 hosts
- Number of caches: 12
- Types of caches (atomicity mode): different (atomic, transactional)
- Types of caches (tiered storage mode): different (onheap without eviction, onheap with eviction,
offheap_tired, offheap_values)
- Types of caches (indexing): different (with and without indexes)
- Types of caches (cache mode): different (partitioned, replicated)
- Backups count: 1
- Duration: 1h

Attachments:
- part of log with the problem (for one of the servers) for config #1
- xml configs and property file that were used to run yardstick for config #1

  was:
During load test grid hung:
{noformat}
[09:22:17,688][WARN ][exchange-worker-#137%null%][GridCachePartitionExchangeManager] Failed
to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=97, minorTopVer=0],
node=0fe0c0f2-65e1-4ad5-a8c9-92db997fcb7d]. Dumpin
g pending objects that might be the cause:
...
{noformat}
See attached file with more info from logs.

Notes:
The issue was also reproduced with IGNITE-3239 fixed. So, IGNITE-3239 doesn't seem to be a
root cause. 

Load test config:
- Benchmark name: IgniteCacheRandomOperationBenchmark
- Operations: put, put_all, get, get_all, invoke, invoke_all, remove, remove_all, put_if_absent,
replace
- Heap: 2Gb for servers, 1Gb for clients
- 48 clients, 48 servers, 6 hosts
- Preload amount: 2.5M
- Key range: 5M
- Number of caches: 12
- Types of caches (atomicity mode): only atomic
- Types of caches (tiered storage mode): different (onheap without eviction, onheap with eviction,
offheap_tired, offheap_values)
- Types of caches (indexing): different (with and without indexes)
- Types of caches (cache mode): partitioned
- Backups count: 1

Attachments:
- part of log with the problem (for one of the servers)
- xml configs and property file that were used to run yardstick


> Grid hung during load test with warning "Failed to wait for partition map exchange"
> -----------------------------------------------------------------------------------
>
>                 Key: IGNITE-3474
>                 URL: https://issues.apache.org/jira/browse/IGNITE-3474
>             Project: Ignite
>          Issue Type: Bug
>    Affects Versions: 1.6
>            Reporter: Ksenia Rybakova
>            Assignee: Semen Boikov
>            Priority: Critical
>         Attachments: 091826_id0_192.168.1.21_part.log, ignite-base-load-config.xml, run-load.properties,
run-load.xml
>
>
> During load test grid hung:
> {noformat}
> [09:22:17,688][WARN ][exchange-worker-#137%null%][GridCachePartitionExchangeManager]
Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=97, minorTopVer=0],
node=0fe0c0f2-65e1-4ad5-a8c9-92db997fcb7d]. Dumpin
> g pending objects that might be the cause:
> ...
> {noformat}
> See attached file with more info from logs.
> Notes:
> The issue was also reproduced with IGNITE-3239 fixed. So, IGNITE-3239 doesn't seem to
be a root cause. 
> Configs:
> 1)
>  - Benchmark name: IgniteCacheRandomOperationBenchmark 
>  - Operations: put, put_all, get, get_all, invoke, invoke_all, remove, remove_all, put_if_absent,
replace
> - Heap: 4Gb for servers, 1Gb for clients
> - 80 clients, 20 servers, 8 hosts
> - Number of caches: 100
> - Preload amount 250K
> - Key range 500K
> - Types of caches (atomicity mode): different (atomic, transactional)
> - Types of caches (tiered storage mode): different (onheap without eviction, onheap with
eviction, offheap_tired, offheap_values)
> - Types of caches (indexing): different (with and without indexes)
> - Types of caches (cache mode): different (partitioned, replicated)
> - Backups count: 1
> - Duration: 1h
> 2) 
> - Benchmark name: IgniteCacheRandomOperationBenchmark 
> - Operations: scan_query, sql_query, continuous_query
> - Heap: 4Gb for servers, 1Gb for clients
> - 20 clients, 80 servers, 8 hosts
> - Number of caches: 12
> - Types of caches (atomicity mode): different (atomic, transactional)
> - Types of caches (tiered storage mode): different (onheap without eviction, onheap with
eviction, offheap_tired, offheap_values)
> - Types of caches (indexing): different (with and without indexes)
> - Types of caches (cache mode): different (partitioned, replicated)
> - Backups count: 1
> - Duration: 1h
> Attachments:
> - part of log with the problem (for one of the servers) for config #1
> - xml configs and property file that were used to run yardstick for config #1



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message