Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <CALH+G9qsrgCxp8C+JQGJ7sJQvfkNCd=nKqYb_VVFidd0GUv0CA@mail.gmail.com>
References: <MWHPR2201MB1551AC32BAAE87F0D3F32331CF4A0@MWHPR2201MB1551.namprd22.prod.outlook.com>
 <CANCAXEGvUFRxCO6b8JnKpd6i3fn4zn9YHLQqDbQgwM2OqwpbRw@mail.gmail.com>
 <CANCAXEGTYL509wQuuG4u-sNysGZqbG9O-ii36JpihXcF+Fc2Xw@mail.gmail.com>
 <MWHPR2201MB15519CDE450EA953F1937A25CF4B0@MWHPR2201MB1551.namprd22.prod.outlook.com>
 <CALH+G9o0nRYShWL4_Ff-rzUm2AATS7iX-r7bPrWqRCt_04rA=Q@mail.gmail.com>
 <MWHPR2201MB1551BA21F3219A5B4FAD03D6CF480@MWHPR2201MB1551.namprd22.prod.outlook.com>
 <MWHPR2201MB1551D2BB3CC550E7B521CD70CF4F0@MWHPR2201MB1551.namprd22.prod.outlook.com>
 <CA+0=VoWUC=0D4gsYgzoTDgfRFT9+aeGqpRSgXVsk4TjrrPC5+A@mail.gmail.com>
 <MWHPR2201MB1551777B50B532731E870CD4CF4C0@MWHPR2201MB1551.namprd22.prod.outlook.com>
 <CABDss3jfUoh0sNw6XHg4U_Ci9+2tkuP-OoWJG_GcV-hW+Rdr4w@mail.gmail.com> <CALH+G9qsrgCxp8C+JQGJ7sJQvfkNCd=nKqYb_VVFidd0GUv0CA@mail.gmail.com>
From: Alexey Goncharuk <alexey.goncharuk@gmail.com>
Date: Tue, 17 Oct 2017 11:43:03 +0300
Message-ID: <CABDss3hCtQoRxQDkhRz5U5h2ke3d2QXcqmE47BqCL75rgrP5Mw@mail.gmail.com>
Subject: Re: Indexing fields of non-POJO cache values
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary="94eb2c0a525081c36e055bba1c69"
archived-at: Tue, 17 Oct 2017 08:43:10 -0000

--94eb2c0a525081c36e055bba1c69
Content-Type: text/plain; charset="UTF-8"

Alexey K., looks like this will require significant changes in H2 (I cannot
find anything on partial indexes there).

Vladimir, any ideas?

2017-10-17 11:35 GMT+03:00 Alexey Kuznetsov <akuznetsov@apache.org>:

> Alexey G.,
>
> >  How these field extractors will be configured. QueryField and
> QueryIndex are
> already quite complex classes.
> > Adding such a closure to configuration would complicate them even
> further.
> May be we can go in "JavaScript" way and pass a string with expression that
> will be parsed and evaluated later on server side.
>
> > How these extractors will interact with future SQL drivers (my current
> guess
> - there is no way to define them in SQL)
>
> AFAIK RDBMS support index on expression.
> For example: https://sqlite.org/expridx.html
>
> Make sense?
>
> On Tue, Oct 17, 2017 at 3:26 PM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
> > I like this idea. In general case, this will not even require
> > deserializing the cache value. Consider a binary tree implementation
> with a
> > binary object node {val, left, right}. In this case, it is impossible to
> > have an index of min or max, but with Andrey's suggestion, these indexes
> > are trivially extracted.
> >
> > Two things to consider:
> >  * How these field extractors will be configured. QueryField and
> QueryIndex
> > are already quite complex classes. Adding such a closure to configuration
> > would complicate them even further.
> >  * How these extractors will interact with future SQL drivers (my current
> > guess - there is no way to define them in SQL)
> >
> > Andrey, can you create a ticket and suggest an API design so we can
> review
> > it?
> >
> > Thanks,
> > AG
> >
> > 2017-10-17 5:44 GMT+03:00 Andrey Kornev <andrewkornev@hotmail.com>:
> >
> > > Of course it does, Dmitriy! However as I suggested below, the feature
> > > should be optional. The current behavior (not requiring user classes on
> > the
> > > server, etc.) would remain the default one.
> > >
> > > Also, please realize that not everyone stores their data as POJOs or
> uses
> > > Ignite as a JDBC source -- the use cases that appear to have been the
> > main
> > > focus of Ignite community lately.
> > >
> > > Payloads with dynamic structures require more advanced mechanisms for
> > > indexing, for example, to avoid the overhead of duplicating the
> indexable
> > > fields as top level fields of the BinaryObjects. In cases where the
> cache
> > > sizes are in tens of millions of entries, the ability to generate index
> > > values on the fly rather than store them, would go a long way in terms
> of
> > > reducing memory utilization.
> > >
> > > In Ignite community finds this feature generally useful, I'd be more
> than
> > > happy to contribute its implementation.
> > >
> > > Regards
> > > Andrey
> > >
> > > ________________________________
> > > From: Dmitriy Setrakyan <dsetrakyan@apache.org>
> > > Sent: Monday, October 16, 2017 6:14 PM
> > > To: dev@ignite.apache.org
> > > Subject: Re: Indexing fields of non-POJO cache values
> > >
> > > On Mon, Oct 16, 2017 at 12:35 PM, Andrey Kornev <
> > andrewkornev@hotmail.com>
> > > wrote:
> > >
> > > > [Crossposting to the dev list]
> > > >
> > > > Alexey,
> > > >
> > > > Yes, something like that, where the "reference"/"alias" is expressed
> > as a
> > > > piece of Java code (as part of QueryEntity definition, perhaps) that
> is
> > > > invoked by Ignite at the cache entry indexing time.
> > > >
> > > > My point is that rather than limiting indexable fields only to
> > predefined
> > > > POJO attributes (or BinaryObject fields) Ignite could adopt a more
> > > general
> > > > approach by allowing users designate an arbitrary piece of code (a
> > > > lambda/closure) to be used as an index value extractor. In such case,
> > the
> > > > current functionality (extracting index values from POJO attributes)
> > > > becomes just a special case that's supported by Ignite out of the
> box.
> > > >
> > >
> > > Andrey, this would require deserialization on the server side. It would
> > > also require that user classes are present on the server side. Both of
> > this
> > > scenarios Ignite tries to avoid.
> > >
> > > Makes sense?
> > >
> >
>
>
>
> --
> Alexey Kuznetsov
>

--94eb2c0a525081c36e055bba1c69--