Mailing-List: contact dev-help@polygene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@polygene.apache.org
MIME-Version: 1.0
Sender: hedhman@gmail.com
In-Reply-To: <f1c885f1-0be6-4f29-ba80-e6c61d080aa5@zest.mail.kapsi.fi>
References: <58E27E03.50601@apache.org> <f1c885f1-0be6-4f29-ba80-e6c61d080aa5@zest.mail.kapsi.fi>
From: Niclas Hedhman <niclas@hedhman.org>
Date: Tue, 4 Apr 2017 16:03:48 +0800
Message-ID: <CADmm+Kcez=mgROqdFy2+PNHp5X95hOUGTLXX=O7u7N0EwHBhFg@mail.gmail.com>
Subject: Re: Sorting out issues with Indexing SQL
To: dev@polygene.apache.org, stanislav.muhametsin@zest.mail.kapsi.fi
Content-Type: multipart/alternative; boundary=94eb2c1a1ee8707378054c52b8b1
archived-at: Tue, 04 Apr 2017 08:04:15 -0000

--94eb2c1a1ee8707378054c52b8b1
Content-Type: text/plain; charset=UTF-8

> Indexing-SQL scans whole application structure on startup to detect all
the visible and indexable entity composite types, and if the things changed
there, it might cause problems.

Oy!  Do you remember How/Where is that done? Because the EntityComposite
type is added to the EntityDescriptor and must be done that way (or the
Entity object with instanceof) and not be checking the Java interfaces.
That could be the root of many problems.

The other change is that identity is now a type and not String, and
somewhere that might cause problems.


Rewrite; Yeah, I have been considering this for a while. And perhaps build
something by leveraging QueryDSL (http://www.querydsl.com/), or at least
borrow internals for it. But I concluded that it was more work than fit
into 3.0 plan. But I would be really happy to get to a situation where we
have "ES backed indexer", which would help a lot going forward. The RDF
implementation is also getting "old"...

Cheers
Niclas

On Tue, Apr 4, 2017 at 3:15 PM, Stanislav Muhametsin <
stanislav.muhametsin@zest.mail.kapsi.fi> wrote:

> Hi!
>
> Hmm yeah, it's been quite a while since I wrote that beast. :D I would do
> a lot of things differently now, but I guess we are stuck with what we have
> (until someone rewrites that).
>
> There was a discussion in October 2016, and I am not sure if the issue
> discussed then has been resolved, or maybe is (partially) the cause for
> errors.
> I will copy-paste my reply I wrote in October at the end of this mail.
>
> Looking at that issue link ( https://issues.apache.org/jira
> /browse/POLYGENE-222 ), there might be something in detection of entity
> composite types - looks like "the first bad commit" was something about
> entity composite types no longer needing to extend EntityComposite
> interface?
> Indexing-SQL scans whole application structure on startup to detect all
> the visible and indexable entity composite types, and if the things changed
> there, it might cause problems.
>
> The other commits are more unfamiliar territory for me - looking at
> attached test report, I think the most important information is the
> standard output.
> There is a bunch of "unsupported property type" messages - are
> associations and manyassociations in Polygene just Property<..> these days?
> That definetly will break things in Indexing-SQL.
> It is a bit hard to follow for me since so many things have changed in
> Polygene now (which is also a good thing - a sign of development!).
>
> Here is the copy-paste from October.
> It seems that the problem was Indexing-SQL not recognizing "Identity" type
> as primitive (something like 'String' or 'Integer').
>
> -------- BEGIN QUOTED MAIL --------
> In "extensions/indexing-sql/src/main/java/org/apache/zest/index
> /sql/support/skeletons/AbstractSQLStartup.java" file, there is
> "initTypes" method.
> You might want to add mapping for Identity.class in this._primitiveTypes
> and jdbcTypes, and also most likely this._customizableTypes.
>
> I say "might" want to, since I have really really vague memories on how
> that worked, and I realized I don't have any PC right now with Java coding
> environment set up.
> The "appendColumnDefinitionsForProperty" method in the same file uses the
> type mappings mentioned above to deduce what kind of column is to be
> created for property, which might be the issue here.
>
> Also, you probably want to modify the "/extensions/indexing-sql/src/
> main/java/org/apache/zest/index/sql/support/common/QNameInfo.java" file,
> so that the detection whether some Java type is primitive is updated to
> include Identity.
> It is done just before setting this._isFinalTypePrimitive.
>
> Let us know if this is of any help to you.
> -------- END QUOTED MAIL --------
>
>
> On 03/04/2017 19:53, Paul Merlin wrote:
>
>> Stan, Niclas,
>>
>> Indexing SQL has been broken for quite some time.
>> See https://issues.apache.org/jira/browse/POLYGENE-222
>>
>> That issue contains the result of bisecting the history to identify when
>> it broke. With the Docker based testing infra it is now very easy to
>> reproduce.
>>
>> Stan, you wrote that beast :)
>> Niclas, one commit of yours broke that beast :)
>>
>> Could you have a look?
>>
>> Thanks!
>>
>> /Paul
>>
>>
>>
>


-- 
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java

--94eb2c1a1ee8707378054c52b8b1--