ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: Default page size must be changed to 4k. Should it be backwards compatible?
Date Fri, 18 Aug 2017 01:18:54 GMT
On Tue, Aug 15, 2017 at 10:09 PM, Serge Puchnin <sergey.puchnin@gmail.com>
wrote:

> #1 option looks more predictable. But it's possible to add a message "for
> better IO performance please migrate to 4K-pages".
>

Agree. Can we update the ticket to make sure that this suggestion is
printed out?


> BR,
> Serge
>
>
> On Wed, 16 Aug 2017 at 03:46, Dmitriy Setrakyan <dsetrakyan@apache.org>
> wrote:
>
> > I like #1 if possible. Of course, if the LFS is empty, then the new
> default
> > should be 4k.
> >
> > On Tue, Aug 15, 2017 at 12:01 PM, Ivan Rakov <ivan.glukos@gmail.com>
> > wrote:
> >
> > > Guys,
> > >
> > > We have benchmarked how checkpoint write speed on SSD disk depends on
> > > various parameters. It became absolutely obvious that using 4K pages in
> > > durable memory instead of 2K brings considerable, significant
> speed-up. I
> > > think, we must set 4K as default page size.
> > > Ticket with detailed explanation: https://issues.apache.org/jira
> > > /browse/IGNITE-5884
> > > Spoiler: it depends on write order and alignment, but writing 4K is at
> > > least *3x faster* than writing 2K when other parameters are the same.
> > >
> > > The question is backwards compatibility. If pageSize is not explicitly
> > set
> > > in user configuration, attempt to start "4k default" Ignite node from
> "2k
> > > default" LFS files will fail with exception:
> > >
> > > class org.apache.ignite.IgniteCheckedException: Failed to verify store
> > >> file (invalid page size) [expectedPageSize=4096, filePageSize=2048]
> > >>     at org.apache.ignite.internal.processors.cache.persistence.file
> > >> .FilePageStore.checkFile(FilePageStore.java:206)
> > >>     at org.apache.ignite.internal.processors.cache.persistence.file
> > >> .FilePageStore.init(FilePageStore.java:416)
> > >>     at org.apache.ignite.internal.processors.cache.persistence.file
> > >> .FilePageStore.read(FilePageStore.java:315)
> > >>     at org.apache.ignite.internal.processors.cache.persistence.file
> > >> .FilePageStoreManager.read(FilePageStoreManager.java:287)
> > >>     at org.apache.ignite.internal.processors.cache.persistence.file
> > >> .FilePageStoreManager.read(FilePageStoreManager.java:272)
> > >>     at org.apache.ignite.internal.processors.cache.persistence.page
> > >> mem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:569)
> > >>     at org.apache.ignite.internal.processors.cache.persistence.page
> > >> mem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:487)
> > >>     at org.apache.ignite.internal.processors.cache.persistence.Grid
> > >> CacheOffheapManager.getOrAllocateCacheMetas(GridCacheOffheap
> > >> Manager.java:515)
> > >>     at org.apache.ignite.internal.processors.cache.persistence.Grid
> > >> CacheOffheapManager.initDataStructures(GridCacheOffheapManager.java:
> 86)
> > >>     at org.apache.ignite.internal.processors.cache.IgniteCacheOffhe
> > >> apManagerImpl.start(IgniteCacheOffheapManagerImpl.java:139)
> > >>     at org.apache.ignite.internal.processors.cache.CacheGroupContex
> > >> t.start(CacheGroupContext.java:868)
> > >>
> > >
> > > I think, we have two options here:
> > >
> > > 1) Obvious and safe - provide silent backwards compatibility. We can
> > > implement a task which will find any LFS file, check its pageSize and
> use
> > > it as default.
> > > 2) Less user-friendly, but in my opinion still better option - crash
> > node,
> > > but make error message more informative. We'll let user know that
> default
> > > pageSize was changed to 4k due to discovered performance boost on most
> > > UNIX-based enviroments with SSD (which is for sure most popular
> > enviroment
> > > among users), and recommend user to migrate to 4K-page LFS. If user
> still
> > > wants to work with 2k pages, he can always set it explicitly in
> > > MemoryConfiguration and start node.
> > >
> > > Thoughts?
> > >
> > > --
> > > Best Regards,
> > > Ivan Rakov
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message