ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Goncharuk <alexey.goncha...@gmail.com>
Subject Re: Indexing fields of non-POJO cache values
Date Tue, 17 Oct 2017 08:26:24 GMT
I like this idea. In general case, this will not even require
deserializing the cache value. Consider a binary tree implementation with a
binary object node {val, left, right}. In this case, it is impossible to
have an index of min or max, but with Andrey's suggestion, these indexes
are trivially extracted.

Two things to consider:
 * How these field extractors will be configured. QueryField and QueryIndex
are already quite complex classes. Adding such a closure to configuration
would complicate them even further.
 * How these extractors will interact with future SQL drivers (my current
guess - there is no way to define them in SQL)

Andrey, can you create a ticket and suggest an API design so we can review
it?

Thanks,
AG

2017-10-17 5:44 GMT+03:00 Andrey Kornev <andrewkornev@hotmail.com>:

> Of course it does, Dmitriy! However as I suggested below, the feature
> should be optional. The current behavior (not requiring user classes on the
> server, etc.) would remain the default one.
>
> Also, please realize that not everyone stores their data as POJOs or uses
> Ignite as a JDBC source -- the use cases that appear to have been the main
> focus of Ignite community lately.
>
> Payloads with dynamic structures require more advanced mechanisms for
> indexing, for example, to avoid the overhead of duplicating the indexable
> fields as top level fields of the BinaryObjects. In cases where the cache
> sizes are in tens of millions of entries, the ability to generate index
> values on the fly rather than store them, would go a long way in terms of
> reducing memory utilization.
>
> In Ignite community finds this feature generally useful, I'd be more than
> happy to contribute its implementation.
>
> Regards
> Andrey
>
> ________________________________
> From: Dmitriy Setrakyan <dsetrakyan@apache.org>
> Sent: Monday, October 16, 2017 6:14 PM
> To: dev@ignite.apache.org
> Subject: Re: Indexing fields of non-POJO cache values
>
> On Mon, Oct 16, 2017 at 12:35 PM, Andrey Kornev <andrewkornev@hotmail.com>
> wrote:
>
> > [Crossposting to the dev list]
> >
> > Alexey,
> >
> > Yes, something like that, where the "reference"/"alias" is expressed as a
> > piece of Java code (as part of QueryEntity definition, perhaps) that is
> > invoked by Ignite at the cache entry indexing time.
> >
> > My point is that rather than limiting indexable fields only to predefined
> > POJO attributes (or BinaryObject fields) Ignite could adopt a more
> general
> > approach by allowing users designate an arbitrary piece of code (a
> > lambda/closure) to be used as an index value extractor. In such case, the
> > current functionality (extracting index values from POJO attributes)
> > becomes just a special case that's supported by Ignite out of the box.
> >
>
> Andrey, this would require deserialization on the server side. It would
> also require that user classes are present on the server side. Both of this
> scenarios Ignite tries to avoid.
>
> Makes sense?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message