ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: memory-only mode for Ignite indexes
Date Mon, 07 May 2018 22:12:43 GMT
Dima, Denis,

Please stoooooooop wanting this! :-)

Guys, there are no in-memory indexes just to be in-memory. Let me briefly
explain what other vendors do and why. There are three approaches:
1) You may create *skip-list* based in-memory index. Goal - speedup
*READS*, not writes. Examples: MS SQL, Couchbase
2) You may create *columnar* in-memory index. Goal - speedup *SCANS* in
OLAP cases, not writes. Example: Oracle [1]
3) You may drop my beloved cache groups and introduce *tablespaces*.instead.
Then you assign data and indexes to different tablespaces and assign them
to *different disks*. Goal - speedup index *WRITES* [2]. Example: all major
vendors. E.g. putting data and indexes on different disks is one of the
main Oracle recommendation in their capacity planning documents.

As you see, nobody afraid of writes. Neither should we do.

Vladimir.

[1]
http://www.oracle.com/technetwork/database/in-memory/overview/twp-oracle-database-in-memory-2245633.pdf
[2] This is one of the reasons why I want to kill cache groups so
desperately.

On Mon, May 7, 2018 at 9:03 PM, Denis Magda <dmagda@apache.org> wrote:

> Truly like the idea of having a separate "indexes in memory only mode"
> regardless of the optimizations Vladimir keeps track of.
>
> In my observations, some database vendors support this as the only mode
> requiring to rebuild the indexes on a restart which confirms that there is
> a demand for this configuration in the industry.
>
> --
> Denis
>
> On Mon, May 7, 2018 at 7:42 AM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
>
> > Agree.
> >
> > On Mon, May 7, 2018 at 5:20 PM, Dmitriy Setrakyan <dsetrakyan@apache.org
> >
> > wrote:
> >
> > > On Mon, May 7, 2018 at 2:09 AM, Vladimir Ozerov <vozerov@gridgain.com>
> > > wrote:
> > >
> > > > Hi Dima,
> > > >
> > > > Update with indexes would definitely be slower than update without
> > them.
> > > > The question is how much slower. For now the slowdown comes mostly
> from
> > > > excessive data page reads ([1] and [2] in my previous email) leading
> to
> > > > page evictions and additional IO. To the contrast, usually only a
> > single
> > > > page write is needed to update an index. Correct index implementation
> > > ([1]
> > > > and [2] from previous email) would eliminate data page reads
> altogether
> > > and
> > > > should give dramatic speedup.
> > > >
> > >
> > > Sounds great. The changes you are suggesting should give us a great
> > > performance boost. Hopefully they should not take too long to
> implement.
> > >
> > > Regardless, once you are done, we should still perform a benchmark with
> > > persistence indexes vs memory-only indexes. If memory-only will be more
> > > than 20% faster, we should still implement it.
> > >
> > > D.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message