ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-7507) Ignite node performance drop during checkpoint start: store metapage eviction causes long checkpoint lock hold time
Date Tue, 23 Jan 2018 16:26:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336011#comment-16336011
] 

ASF GitHub Bot commented on IGNITE-7507:
----------------------------------------

GitHub user dspavlov opened a pull request:

    https://github.com/apache/ignite/pull/3423

    IGNITE-7507: Ignite node performance drop during checkpoint start

    Store metapage eviction causes long checkpoint lock hold time

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gridgain/apache-ignite ignite-7507

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/ignite/pull/3423.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3423
    
----
commit 0d557e567d7b770f2c91306071af9b849f0e66b3
Author: dspavlov <dpavlov.spb@...>
Date:   2018-01-23T16:20:43Z

    IGNITE-7507: Ignite node performance drop during checkpoint start: store metapage eviction
causes long checkpoint lock hold time

----


> Ignite node performance drop during checkpoint start: store metapage eviction causes
long checkpoint lock hold time
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: IGNITE-7507
>                 URL: https://issues.apache.org/jira/browse/IGNITE-7507
>             Project: Ignite
>          Issue Type: Bug
>          Components: persistence
>            Reporter: Dmitriy Pavlov
>            Assignee: Dmitriy Pavlov
>            Priority: Major
>             Fix For: 2.5
>
>
> Store metadata Page eviction becomes very expensive operation during checkpoint start.
> These pages reads hands ignite node until metadata will be loaded from disk.
> Following store (paritition) metapages:
> - Partition Metadata Page
> - Freelist Meta Page
> - Partition Counters IO
> required during execution of saveStoreMetadata() & markCheckpointBegin()
> If this page is not available in memory, it is loaded from disk.
> But such loads are done while holding checkpointLock (in write mode).
> Example of timing:
> - checkpointLockWait=75ms, checkpointLockHoldTime=2653ms, pages=696120
> All this time worker threads are not able to put any data to any cache.
> It is required to avoid eviction of such pages (evict it with lowest priority than dirty
page).
> (Full stacktrace)	
> {noformat} db-checkpoint-thread-#40%checkpoint.IgniteMassLoadSandboxTest1% Id=63 WAITING

> 	
> at sun.misc.Unsafe.park(Native Method)
> 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> 	at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> 	at org.apache.ignite.internal.util.future.GridFutureAdapter.getUninterruptibly(GridFutureAdapter.java:145)
> 	at org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIO.read(AsyncFileIO.java:95)
> 	at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:324)
> 	at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:306)
> 	at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:291)
> 	at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:656)
> 	at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:576)
> 	at org.apache.ignite.internal.processors.cache.persistence.DataStructure.acquirePage(DataStructure.java:130)
> 	at org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.saveMetadata(PagesList.java:301)
> 	at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.saveStoreMetadata(GridCacheOffheapManager.java:196)
> 	at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.onCheckpointBegin(GridCacheOffheapManager.java:168)
> 	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:3022)
> 	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:2719)
> 	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:2644)
> 	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> 	at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message