ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Default page size must be changed to 4k. Should it be backwards compatible?
Date Fri, 18 Aug 2017 23:12:35 GMT
Added Ignite Persistence related section to the “Performance Tips” docs and mentioned this
suggestion there:
https://apacheignite.readme.io/docs/performance-tips#tune-ignite-native-persistence

—
Denis

> On Aug 17, 2017, at 6:18 PM, Dmitriy Setrakyan <dsetrakyan@apache.org> wrote:
> 
> On Tue, Aug 15, 2017 at 10:09 PM, Serge Puchnin <sergey.puchnin@gmail.com>
> wrote:
> 
>> #1 option looks more predictable. But it's possible to add a message "for
>> better IO performance please migrate to 4K-pages".
>> 
> 
> Agree. Can we update the ticket to make sure that this suggestion is
> printed out?
> 
> 
>> BR,
>> Serge
>> 
>> 
>> On Wed, 16 Aug 2017 at 03:46, Dmitriy Setrakyan <dsetrakyan@apache.org>
>> wrote:
>> 
>>> I like #1 if possible. Of course, if the LFS is empty, then the new
>> default
>>> should be 4k.
>>> 
>>> On Tue, Aug 15, 2017 at 12:01 PM, Ivan Rakov <ivan.glukos@gmail.com>
>>> wrote:
>>> 
>>>> Guys,
>>>> 
>>>> We have benchmarked how checkpoint write speed on SSD disk depends on
>>>> various parameters. It became absolutely obvious that using 4K pages in
>>>> durable memory instead of 2K brings considerable, significant
>> speed-up. I
>>>> think, we must set 4K as default page size.
>>>> Ticket with detailed explanation: https://issues.apache.org/jira
>>>> /browse/IGNITE-5884
>>>> Spoiler: it depends on write order and alignment, but writing 4K is at
>>>> least *3x faster* than writing 2K when other parameters are the same.
>>>> 
>>>> The question is backwards compatibility. If pageSize is not explicitly
>>> set
>>>> in user configuration, attempt to start "4k default" Ignite node from
>> "2k
>>>> default" LFS files will fail with exception:
>>>> 
>>>> class org.apache.ignite.IgniteCheckedException: Failed to verify store
>>>>> file (invalid page size) [expectedPageSize=4096, filePageSize=2048]
>>>>>    at org.apache.ignite.internal.processors.cache.persistence.file
>>>>> .FilePageStore.checkFile(FilePageStore.java:206)
>>>>>    at org.apache.ignite.internal.processors.cache.persistence.file
>>>>> .FilePageStore.init(FilePageStore.java:416)
>>>>>    at org.apache.ignite.internal.processors.cache.persistence.file
>>>>> .FilePageStore.read(FilePageStore.java:315)
>>>>>    at org.apache.ignite.internal.processors.cache.persistence.file
>>>>> .FilePageStoreManager.read(FilePageStoreManager.java:287)
>>>>>    at org.apache.ignite.internal.processors.cache.persistence.file
>>>>> .FilePageStoreManager.read(FilePageStoreManager.java:272)
>>>>>    at org.apache.ignite.internal.processors.cache.persistence.page
>>>>> mem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:569)
>>>>>    at org.apache.ignite.internal.processors.cache.persistence.page
>>>>> mem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:487)
>>>>>    at org.apache.ignite.internal.processors.cache.persistence.Grid
>>>>> CacheOffheapManager.getOrAllocateCacheMetas(GridCacheOffheap
>>>>> Manager.java:515)
>>>>>    at org.apache.ignite.internal.processors.cache.persistence.Grid
>>>>> CacheOffheapManager.initDataStructures(GridCacheOffheapManager.java:
>> 86)
>>>>>    at org.apache.ignite.internal.processors.cache.IgniteCacheOffhe
>>>>> apManagerImpl.start(IgniteCacheOffheapManagerImpl.java:139)
>>>>>    at org.apache.ignite.internal.processors.cache.CacheGroupContex
>>>>> t.start(CacheGroupContext.java:868)
>>>>> 
>>>> 
>>>> I think, we have two options here:
>>>> 
>>>> 1) Obvious and safe - provide silent backwards compatibility. We can
>>>> implement a task which will find any LFS file, check its pageSize and
>> use
>>>> it as default.
>>>> 2) Less user-friendly, but in my opinion still better option - crash
>>> node,
>>>> but make error message more informative. We'll let user know that
>> default
>>>> pageSize was changed to 4k due to discovered performance boost on most
>>>> UNIX-based enviroments with SSD (which is for sure most popular
>>> enviroment
>>>> among users), and recommend user to migrate to 4K-page LFS. If user
>> still
>>>> wants to work with 2k pages, he can always set it explicitly in
>>>> MemoryConfiguration and start node.
>>>> 
>>>> Thoughts?
>>>> 
>>>> --
>>>> Best Regards,
>>>> Ivan Rakov
>>>> 
>>>> 
>>> 
>> 


Mime
View raw message