ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy Pavlov (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IGNITE-7730) Improve WAL history size documentation
Date Tue, 30 Oct 2018 11:04:00 GMT

     [ https://issues.apache.org/jira/browse/IGNITE-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Dmitriy Pavlov resolved IGNITE-7730.
    Resolution: Done

According there is a new approach to specify WAL history size in bytes
there is no need to deep specification calculation algorithm. 

The algorithm is specified in wiki https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-Estimatingdiskspace
for an advanced user/Ignite developer

> Improve WAL history size documentation
> --------------------------------------
>                 Key: IGNITE-7730
>                 URL: https://issues.apache.org/jira/browse/IGNITE-7730
>             Project: Ignite
>          Issue Type: Task
>          Components: documentation
>    Affects Versions: 2.1
>            Reporter: Dmitriy Pavlov
>            Assignee: Dmitriy Pavlov
>            Priority: Major
>             Fix For: 2.7
> Until IGNITE-6552 is not implemented, we have only ability to configure WAL hist. size
in checkpoints.
> It is needed to improve description for this parameter.
> I've added draft notes to wiki [https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-Estimatingdiskspace]
>  about ways how wer can estimate WAL sizes without exact bytes/time specification:
> {panel}
> WAL Work max used size: walSegmentSize * walSegments = 640Mb (default)
> in case Default WAL mode - this size is used always,
> in case other modes best case is 1 segment * walSegmentSize
> WAL Work+WAL Archive max size may be estimated by
>  1. average load or
>  2. by maximum size.
>  1st way is applicable if checkpoints are triggered mostly by timer trigger. 
>  Wal size = 2*Average load(bytes/sec) * trigger interval (sec) * walHistSize (number
of checkpoints)
>  Where 2 multiplier coming from physical & logical WAL Records.
> 2nd way: Checkpoint is triggered by segments max dirty pages percent. Use persisted data
regions max sizes:
>  sum(Max configured DataRegionConfiguration.maxSize) * 75% - est. maximum data volume
to be writen on 1 checkpoint.
>  Overall WAL size (before archiving) = 2* est. data volume * walHistSize = 1,5 * sum(DataRegionConfiguration.maxSize)
* walHistSize
> Note applying WAL compressor may significiantly reduce archive size.
> {panel}
> One more note from [~ivan.glukos] on dev.list we need to include. It is answer to question
how user can determine if segment from archive folder can be safely removed: 
> {quote}By the way: WAL compression is already implemented that way. If there
>  are any ".zip" segments in archive dir, they are free to delete.
>  This can be a safe workaround for users who experience lack of free
>  space - just delete compressed segments. We should mention it in
>  documentation for 2.4 release.
> {quote}

This message was sent by Atlassian JIRA

View raw message