polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niclas Hedhman <nic...@hedhman.org>
Subject Re: Sorting out issues with Indexing SQL
Date Tue, 04 Apr 2017 10:17:55 GMT
Thanks, I will try to dig into this. Worst case, we take indexing-sql off
the release specification.

On Tue, Apr 4, 2017 at 5:26 PM, Stanislav Muhametsin <
stanislav.muhametsin@zest.mail.kapsi.fi> wrote:

> On 4.4.2017 11:03, Niclas Hedhman wrote:
>
>> Indexing-SQL scans whole application structure on startup to detect all
>>>
>> the visible and indexable entity composite types, and if the things
>> changed
>> there, it might cause problems.
>>
>> Oy!  Do you remember How/Where is that done? Because the EntityComposite
>> type is added to the EntityDescriptor and must be done that way (or the
>> Entity object with instanceof) and not be checking the Java interfaces.
>> That could be the root of many problems.
>>
>
> It is "constructApplicationInfo" method in "extensions/indexing-sql/src/m
> ain/java/org/apache/polygene/index/sql/support/skeletons/Abs
> tractSQLStartup.java".
> However, I am looking at it (via GitHub - still no Java IDE here), and it
> doesn't _seem_ to use direct type check - it uses EntityDescriptors instead.
> You still might wanna make sure that it actually works as needed.
>
> Unfortunately, I couldn't find the code which emits "unsupported property
> type" messages (no text-content-search in Github :( ), but putting a
> breakpoint in there might reveal something crucial.
> I am very certain that I didn't get any of those when I got Indexing-SQL
> working back in the days.
>
>
>
>
>> The other change is that identity is now a type and not String, and
>> somewhere that might cause problems.
>>
>>
>> Rewrite; Yeah, I have been considering this for a while. And perhaps build
>> something by leveraging QueryDSL (http://www.querydsl.com/), or at least
>> borrow internals for it. But I concluded that it was more work than fit
>> into 3.0 plan. But I would be really happy to get to a situation where we
>> have "ES backed indexer", which would help a lot going forward. The RDF
>> implementation is also getting "old"...
>>
>> Cheers
>> Niclas
>>
>> On Tue, Apr 4, 2017 at 3:15 PM, Stanislav Muhametsin <
>> stanislav.muhametsin@zest.mail.kapsi.fi> wrote:
>>
>> Hi!
>>>
>>> Hmm yeah, it's been quite a while since I wrote that beast. :D I would do
>>> a lot of things differently now, but I guess we are stuck with what we
>>> have
>>> (until someone rewrites that).
>>>
>>> There was a discussion in October 2016, and I am not sure if the issue
>>> discussed then has been resolved, or maybe is (partially) the cause for
>>> errors.
>>> I will copy-paste my reply I wrote in October at the end of this mail.
>>>
>>> Looking at that issue link ( https://issues.apache.org/jira
>>> /browse/POLYGENE-222 ), there might be something in detection of entity
>>> composite types - looks like "the first bad commit" was something about
>>> entity composite types no longer needing to extend EntityComposite
>>> interface?
>>> Indexing-SQL scans whole application structure on startup to detect all
>>> the visible and indexable entity composite types, and if the things
>>> changed
>>> there, it might cause problems.
>>>
>>> The other commits are more unfamiliar territory for me - looking at
>>> attached test report, I think the most important information is the
>>> standard output.
>>> There is a bunch of "unsupported property type" messages - are
>>> associations and manyassociations in Polygene just Property<..> these
>>> days?
>>> That definetly will break things in Indexing-SQL.
>>> It is a bit hard to follow for me since so many things have changed in
>>> Polygene now (which is also a good thing - a sign of development!).
>>>
>>> Here is the copy-paste from October.
>>> It seems that the problem was Indexing-SQL not recognizing "Identity"
>>> type
>>> as primitive (something like 'String' or 'Integer').
>>>
>>> -------- BEGIN QUOTED MAIL --------
>>> In "extensions/indexing-sql/src/main/java/org/apache/zest/index
>>> /sql/support/skeletons/AbstractSQLStartup.java" file, there is
>>> "initTypes" method.
>>> You might want to add mapping for Identity.class in this._primitiveTypes
>>> and jdbcTypes, and also most likely this._customizableTypes.
>>>
>>> I say "might" want to, since I have really really vague memories on how
>>> that worked, and I realized I don't have any PC right now with Java
>>> coding
>>> environment set up.
>>> The "appendColumnDefinitionsForProperty" method in the same file uses
>>> the
>>> type mappings mentioned above to deduce what kind of column is to be
>>> created for property, which might be the issue here.
>>>
>>> Also, you probably want to modify the "/extensions/indexing-sql/src/
>>> main/java/org/apache/zest/index/sql/support/common/QNameInfo.java" file,
>>> so that the detection whether some Java type is primitive is updated to
>>> include Identity.
>>> It is done just before setting this._isFinalTypePrimitive.
>>>
>>> Let us know if this is of any help to you.
>>> -------- END QUOTED MAIL --------
>>>
>>>
>>> On 03/04/2017 19:53, Paul Merlin wrote:
>>>
>>> Stan, Niclas,
>>>>
>>>> Indexing SQL has been broken for quite some time.
>>>> See https://issues.apache.org/jira/browse/POLYGENE-222
>>>>
>>>> That issue contains the result of bisecting the history to identify when
>>>> it broke. With the Docker based testing infra it is now very easy to
>>>> reproduce.
>>>>
>>>> Stan, you wrote that beast :)
>>>> Niclas, one commit of yours broke that beast :)
>>>>
>>>> Could you have a look?
>>>>
>>>> Thanks!
>>>>
>>>> /Paul
>>>>
>>>>
>>>>
>>>>
>>
>


-- 
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message