ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Rakov <ivan.glu...@gmail.com>
Subject Re: WAL Archive Issue
Date Tue, 13 Feb 2018 19:06:44 GMT
Yakov,

This will work. However, I expect performance degradation with this 
change. Disk storage has a limited number of I/O operations per second 
on hardware level. List of already existing disk I/O activities (writing 
to WAL work dir, copying from WAL work dir to WAL archive dir, writing 
partition files during checkpoint) will be updated with a new one - 
copying from WAL work dir to temp dir.

Best Regards,
Ivan Rakov

On 13.02.2018 21:35, Yakov Zhdanov wrote:
> Ivan,
>
> I do not want to create new files. As far as I know, now we copy segments
> to archive dir before they get checkpointed. What I suggest is to copy them
> to a temp dir under wal directory and then move to archive. In my
> understanding at the time we copy the files to a temp folder all changes to
> them are already fsynced.
>
> Correct?
>
> Yakov Zhdanov,
> www.gridgain.com
>
> 2018-02-13 21:29 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com>:
>
>> Yakov,
>>
>> I see the only one problem with your suggestion - number of
>> "uncheckpointed" segments is potentially unlimited.
>> Right now we have limited number (10) of file segments with immutable
>> names in WAL "work" directory. We have to keep this approach due to known
>> bug in XFS - fsync time is nearly twice bigger for recently created files.
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>> On 13.02.2018 21:22, Yakov Zhdanov wrote:
>>
>>> I meant we still will be copying segment once and then will be moving it
>>> to
>>> archive which should not affect file system much.
>>>
>>> Thoughts?
>>>
>>> --Yakov
>>>
>>> 2018-02-13 21:19 GMT+03:00 Yakov Zhdanov <yzhdanov@apache.org>:
>>>
>>> Alex,
>>>> I remember we had some confusing behavior for WAL archive when archived
>>>> segments were required for successful recovery.
>>>>
>>>> Is issue still present?
>>>>
>>>> If yes, what if we copy "uncheckpointed" segments to a directory under
>>>> wal
>>>> directory and then move the segments to archive after checkpoint? Will
>>>> this
>>>> work?
>>>>
>>>> Thanks!
>>>>
>>>> --Yakov
>>>>
>>>>


Mime
View raw message