ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Pavlov <dpavlov....@gmail.com>
Subject Re: Page replacement policy improvements (when persistent is enabled)
Date Thu, 16 Aug 2018 10:25:43 GMT
Hi Igniters,

I feel monitoring will provide us with the necessary knowledge about how to

I have no idea how region separation can help with performance. But it
could be an additional challenge for users to correctly configure separate

Dmitriy Pavlov

чт, 16 авг. 2018 г. в 12:45, Dmitriy Setrakyan <dsetrakyan@apache.org>:

> On Thu, Aug 16, 2018 at 2:27 AM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
> > Dima,
> >
> > None database I know use separate regions for index pages due to the
> reason
> > I expressed above. Instead, they split all pages into two groups - hot
> and
> > cold. With certain rules on how to move pages inside and between these
> > groups. None of these algorithms are complex enough. In fact, they are
> > pretty straightforward and battle-tested. When implemented properly it
> > doesn't matter whether the page is index page or data page. The only
> thing
> > that matter is how often it is accessed. This is critical piece that we
> > lack in the product - our policy is called "random *LRU*", while in
> reality
> > is not LRU at all.
> >
> Aerospike keeps index pages in memory, so there is at least one database
> that does that. I am sure if we research around, there will be more.
> > As far as index pages replacement we do not know whether this is problem
> at
> > all. We heard some complaints that it might be a problem. But we didn't
> see
> > any proofs (thanks to lack of monitoring) and even if this is a problem,
> we
> > do not understand how severe it is. May be it adds 1% overhead and can be
> > ignored for years, may be it adds 1000% overhead and must be fixed
> > immediately.
> >
> I remember one test case with a potential user where we had to change our
> eviction algorithm to avoid evicting index pages and because of that
> improved performance by about 10x.
> > This is sensitive piece of a product. Let's use objective data, not
> > assumptions.
> >
> I agree. The difference is that we need a solution in the mean time. I am
> suggesting a very straight forward approach that can be added fairly
> quickly and will solve majority performance problems associated with index
> page eviction. Once we have that, we can take our time and investigate
> further.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message