ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy Govorukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-11641) Server node copies a lot of WAL files in WAL archive after restart
Date Tue, 16 Apr 2019 14:51:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819105#comment-16819105
] 

Dmitriy Govorukhin commented on IGNITE-11641:
---------------------------------------------

[~mstepachev] Thanks for the review. I fixed your comments.

> Server node copies a lot of WAL files in WAL archive after restart
> ------------------------------------------------------------------
>
>                 Key: IGNITE-11641
>                 URL: https://issues.apache.org/jira/browse/IGNITE-11641
>             Project: Ignite
>          Issue Type: Bug
>            Reporter: Dmitriy Govorukhin
>            Assignee: Dmitriy Govorukhin
>            Priority: Major
>             Fix For: 2.8
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Pre-condition: PDS is enabled, wal_path and wal_archive_path are set in config file.
> 1. Cluster is up and running. Some data uploaded into caches.
> 2. Start load to generate a lot of files in wal archive (more than files in wal directory).
> 3. Stop some node and delete all files from wal archive.
> 4. Start node.
> In this case node copies WAL files from WAL dir into wal archive dir again and again
until the amount of files will be the same it was in wal archive before stop.
> Here is information from server node log
> {code}
> 10:10:17,054][INFO][main][GridCacheDatabaseSharedManager] Restoring partition state for
local groups.
> [10:10:18,522][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal]
> [10:10:18,523][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=1, segIdx=1, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal]
> [10:10:20,631][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal]
> [10:10:20,632][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=2, segIdx=2, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal]
> [10:10:23,276][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal]
> [10:10:23,276][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=3, segIdx=3, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal]
> [10:10:23,995][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal]
> [10:10:23,996][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=4, segIdx=4, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal]
> [10:10:24,644][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal]
> [10:10:24,645][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=5, segIdx=5, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal]
> [10:10:25,301][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal]
> [10:10:25,301][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=6, segIdx=6, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal]
> [10:10:26,043][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal]
> [10:10:26,044][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=7, segIdx=7, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal]
> [10:10:26,768][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal]
> [10:10:26,769][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=8, segIdx=8, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal]
> [10:10:27,600][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal]
> [10:10:27,601][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=9, segIdx=9, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal]
> [10:10:28,315][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal]
> [10:10:28,316][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=10, segIdx=0, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000010.wal]
> [10:10:29,092][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000010.wal]
> [10:10:29,092][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=11, segIdx=1, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000011.wal]
> [10:10:29,908][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000011.wal]
> [10:10:29,908][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=12, segIdx=2, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000012.wal]
> [10:10:30,716][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000012.wal]
> [10:10:30,716][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=13, segIdx=3, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000013.wal]
> [10:10:31,402][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000013.wal]
> [10:10:31,403][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=14, segIdx=4, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000014.wal]
> [10:10:32,052][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000014.wal]
> [10:10:32,053][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Starting to
copy WAL segment [absIdx=15, segIdx=5, origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000015.wal]
> [10:10:32,679][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager] Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000015.wal]
> [10:10:33,986][INFO][main][GridCacheDatabaseSharedManager] Finished restoring partition
state for local groups [groupsProcessed39partitionsProcessed=15056, time=16928ms]
> [10:10:34,269][INFO][main][TcpDiscoverySpi] Connection check threshold is calculated:
2000
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message