ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Kornev <andrewkor...@hotmail.com>
Subject Indexing fields of non-POJO cache values
Date Mon, 16 Oct 2017 19:35:47 GMT
[Crossposting to the dev list]


Yes, something like that, where the "reference"/"alias" is expressed as a piece of Java code
(as part of QueryEntity definition, perhaps) that is invoked by Ignite at the cache entry
indexing time.

My point is that rather than limiting indexable fields only to predefined POJO attributes
(or BinaryObject fields) Ignite could adopt a more general approach by allowing users designate
an arbitrary piece of code (a lambda/closure) to be used as an index value extractor. In such
case, the current functionality (extracting index values from POJO attributes) becomes just
a special case that's supported by Ignite out of the box.

This would really help in cases (like mine) where the cache values are non-POJO entities.

From: Alexey Kuznetsov <akuznetsov@apache.org>
Sent: Thursday, October 12, 2017 5:53 PM
To: user@ignite.apache.org
Subject: Re: Indexing fields of non-POJO cache values

Just as idea.

What if we can to declare a kind of "references" or "aliases" for fields in such cases?
And this will help us to avoid duplication of data.

For example in JavaScript I could (almost on the fly) declare getters and setters that could
be as aliases for my data.

On Fri, Oct 13, 2017 at 12:39 AM, Andrey Kornev <andrewkornev@hotmail.com<mailto:andrewkornev@hotmail.com>>
Hey Andrey,

Thanks for your reply!

We've been using a slightly different approach, where we extract the values of the indexable
leaf nodes and store them as individual fields of the binary object along with the serialized
tree itself. Then we configure the cache to use those fields as QueryEntities. It works fine
and this way we avoid using joins in our queries.

However an obvious drawback of such approach is data duplication. We end up with three copies
of a field value:

1) the leaf node of the tree,
2) the field of the binary object, and
3) Ignite index

I was hoping that there may be a better way to achieve this. In particular I'd like to avoid
storing the value as a field of a binary object (copy #2).

One possible (and elegant) approach to solving this problem would be to introduce a way to
specify a method (or a closure) for a QueryEntity in addition to currently supported BinaryObject
field/POJO attribute.


From: Andrey Mashenkov <andrey.mashenkov@gmail.com<mailto:andrey.mashenkov@gmail.com>>
Sent: Thursday, October 12, 2017 6:25 AM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: Indexing fields of non-POJO cache values


Another way here is to implement your own query engine by extending IndexingSPI interface,
which looks much more complicated.

On Thu, Oct 12, 2017 at 4:23 PM, Andrey Mashenkov <andrey.mashenkov@gmail.com<mailto:andrey.mashenkov@gmail.com>>

There is no way to index such data as is. To index data you need to have entry_field<->column
mapping configured.
As a workaround here, leaves can be stored in cache as values.

E.g. you can have a separate cache to index leaf nodes, where entries will have 2 fields:
"original tree key" field and "leaf node value" indexed field.
So, you will be able to query serialized tree-like structures via SQL query with JOIN condition
on  "original tree key" and WHERE condition on "leaf node value" field.
Obviously, you will need to implement intermediate logic to keep data of both caches consistent.

On Wed, Oct 11, 2017 at 9:40 PM, Andrey Kornev <andrewkornev@hotmail.com<mailto:andrewkornev@hotmail.com>>

Consider the following use case: my cache values are a serialized tree-like structure (as
opposed to a POJO). The leaf nodes of the tree are Java primitives. Some of the leaf nodes
are used by the queries and should be indexed.

What are my options for indexing such data?


Best regards,
Andrey V. Mashenkov

Best regards,
Andrey V. Mashenkov

Alexey Kuznetsov

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message