ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy Pavlov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (IGNITE-7533) Throttle writting threads according fsync progress and checkpoint writting speed instead of region fill
Date Thu, 01 Feb 2018 12:28:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-7533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347192#comment-16347192
] 

Dmitriy Pavlov edited comment on IGNITE-7533 at 2/1/18 12:27 PM:
-----------------------------------------------------------------

Warning is also generated each 10secs if throttling is applied and affects current mark process
more than 20%
{noformat}
Throttling is applied to page modifications [percentOfPartTime=0,20, markDirty=7905 pages/sec,
checkpointWrite=32945 pages/sec, estIdealMarkDirty=9869 pages/sec, curDirty=0,73, maxDirty=0,69,
avgParkTime=203099 ns, pages: (total=373443, evicted=10891, written=360734, synced=300052)]
{noformat}


was (Author: dpavlov):
Warning is also generated each 10secs if throttling is applied and affects current mark process
more than 20%
{noformat}
Throttling is applied to page mark [weight=0,20, mark=7905 pages/sec, checkpointWrite=32945
pages/sec, estIdealMark=9869 pages/sec, curDirty=0,73, targetDirty=0,69, avgParkTime=203099
ns, pages: (total=373443, evicted=10891, written=360734, synced=300052)]
{noformat}

> Throttle writting threads according fsync progress and checkpoint writting speed instead
of region fill
> -------------------------------------------------------------------------------------------------------
>
>                 Key: IGNITE-7533
>                 URL: https://issues.apache.org/jira/browse/IGNITE-7533
>             Project: Ignite
>          Issue Type: Improvement
>          Components: persistence
>    Affects Versions: 2.3
>            Reporter: Dmitriy Pavlov
>            Assignee: Dmitriy Pavlov
>            Priority: Major
>             Fix For: 2.5
>
>         Attachments: image (2).png, imageRatioBased.png, imageSpeedBased.png
>
>
> Throttling implemented under IGNITE-6334 is based on region fill percentage (ditry pages
ratio) and current checkpoint progress.
> But actual progress of writting is based on write operation complete, but not on fsync()
complete.
> Suppose following stage of CP is currently running: most of data is being written and
fsync is started. Fsync on experiments requires more time than write, but throttling is disabled
for that stage. There is enough time to unthrottled grid to fill remaining 16% of clear pages
to get sufficient 75% of dirty pages and writes stops.
> Fsync progress is to be included in checkpoint progress, but actual fsync progress reported
by OS is irregular both on Linux and Windows. See picture, green line is fsync progress, and
yellow is write complete.
> Because fsync progress reported is not regular (the differences are 3-4 orders of magnitude)
it is suggested to implement new speed based throttling policy.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message