ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-7182) Slow sorting of pages collection on checkpoint begin can cause zero dropdown even with throttling enabled
Date Thu, 14 Dec 2017 13:49:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-7182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290852#comment-16290852
] 

ASF GitHub Bot commented on IGNITE-7182:
----------------------------------------

GitHub user dspavlov opened a pull request:

    https://github.com/apache/ignite/pull/3229

    IGNITE-7182: Checkpoint pages: optimize and parallel sorting, fast start

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gridgain/apache-ignite ignite-7182

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/ignite/pull/3229.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3229
    
----
commit 45f6340a518d8580c96614e0c12eb379a2c6202d
Author: dpavlov <dpavlov@gridgain.com>
Date:   2017-12-13T15:43:39Z

    IGNITE-7182: async sorting of CP pages first variant

commit c088390173d04e5be57d3f91bc602733466702da
Author: dpavlov <dpavlov@gridgain.com>
Date:   2017-12-13T19:23:45Z

    IGNITE-7182: lazy transfer of quick-sorted values to async checkpointer, WIP

commit 7643854c87d81a18239489bf75047add6168929a
Author: dpavlov <dpavlov@gridgain.com>
Date:   2017-12-14T12:12:35Z

    IGNITE-7182: lazy transfer of quick-sorted values to async checkpointer, WIP

commit dd4894f30f47ba86c505bc42e650de44ab6cee57
Author: dpavlov <dpavlov@gridgain.com>
Date:   2017-12-14T12:59:01Z

    IGNITE-7182: checkpoint scope class instead of T2

----


> Slow sorting of pages collection on checkpoint begin can cause zero dropdown even with
throttling enabled
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: IGNITE-7182
>                 URL: https://issues.apache.org/jira/browse/IGNITE-7182
>             Project: Ignite
>          Issue Type: Improvement
>          Components: persistence
>    Affects Versions: 2.3
>            Reporter: Ivan Rakov
>            Assignee: Dmitriy Pavlov
>             Fix For: 2.4
>
>
> Tests show that GridCacheDatabaseSharedManager#splitAndSortCpPagesIfNeeded call can last
several seconds on nodes with big amount of memory (>10GB). We should optimize sorting
algorithm, possibly making it multithreaded.
> Another option to make pages write throttling more smooth is to get rid of this heuristic:
> {noformat}
>                 // Starting with 0.05 to avoid throttle right after checkpoint start
>                 // 7/12 is maximum ratio of dirty pages
>                 dirtyRatioThreshold = (dirtyRatioThreshold * 0.95 + 0.05) * 7 / 12;
> {noformat}
> We should replace "magic" lower bound 0.05 * 7 / 12 with the real percentage of dirty
pages at the moment of GridCacheDatabaseSharedManager.Checkpointer#markCheckpointBegin call
return.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message