syncope-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Francesco Chicchiriccò <ilgro...@apache.org>
Subject Re: AnyObjects query
Date Tue, 04 Jul 2017 14:04:00 GMT
On 04/07/2017 15:57, Sergey Beryozkin wrote:
> I did some experiments in CXF:
>
> https://github.com/apache/cxf/blob/master/rt/rs/extensions/search/src/test/java/org/apache/cxf/jaxrs/ext/search/sql/SQLHierarchicalQueryTest.java

>
>
> Indeed, making it work at the generic level, without the typed model, 
> requires some flexibility at the code level :-), with the proper 
> customization supported if needed.
>
> For example, in the CXF code, the name of the joining key is deduced 
> at the moment, given that the primary table is 'printers' and the 
> joining one is 'cartridges' then it is assumed that a 'printers' table 
> has an 'id' primary key while 'cartridges' has a 'printer_id' foreign 
> key ('printers' minus 's' + "_id").
>
> I think the custom visitor optimized around the specific data model 
> can do it much better :-)

Hi Sergey,
thanks for your experiments.

Please consider that we don't even have two distinct tables 'printers' 
and 'cartridges' but a single "AnyObjects" table, where attributes and 
their values are stored into separate tables, and the actual search is 
performed against the views

https://github.com/apache/syncope/blob/2_0_X/core/persistence-jpa/src/main/resources/views.xml#L129-L199

depending on the actual arguments of the requested search.

A said, enhancing our custom visitor (e.g. 
org.apache.syncope.core.persistence.api.search.SearchCondVisitor) to 
cope with such searches is definitely not a trivial task.

Regards.

> On 29/06/17 09:49, Sergey Beryozkin wrote:
>> Yes, as far as the convention is concerned, one would express it
>> as
>>
>> GET
>>
>> /printers?_s=cartridges.colour=blue
>>
>> then at the the next stage it depends if a typed model already 
>> exists, if yes, then it can work OOB with for ex JPA2 CXF visitor.
>> In case of Syncope the model is dynamic, hence the custom Syncope 
>> visitor deals with a string such 'cartridges.colour=blue' itself but 
>> at the moment it does not attach any significance to a '.'.
>>
>> It will need to be enhanced for it to process '.' and build a more 
>> sophisticated Natve JPA query. It is doable, I agree with Francesco 
>> it will be more involved...
>>
>> I propose that at least we create a JIRA to track the enhancement 
>> request.
>>
>> I can experiment at the CXF level to enhance its SQLPrinterVisitor to 
>> see what sort of processing can be required, I'm certain it can be done
>>
>> Sergey
>>
>> On 28/06/17 17:34, Colm O hEigeartaigh wrote:
>>> Thanks for the feedback guys! Let me just expand a bit on the 
>>> motivation
>>> behind my previous example....
>>>
>>> Let's say I'm managing hundreds of printers each of which have a
>>> relationship to a cartridge (of which there are many hundreds) with a
>>> colour attribute. I want to find the printers with a blue cartridge.
>>>
>>> So I first make a search for a list of "blue" cartridges. Then I 
>>> search for
>>> the printers that have a relationship with these cartridges. The 
>>> problem is
>>> on the second search I end up with a ginormous search expression
>>> "relationships%3D%3D7db4512-ad25-40e8-bc78-63ad25c0e894%2C%24relationships%3D%3D16dc6acd-6.....".

>>>
>>> " that could be an invalid URL.
>>>
>>> Is there a better way of handling it than this?
>>>
>>> Colm.
>>>
>>> On Wed, Jun 28, 2017 at 10:52 AM, Sergey Beryozkin 
>>> <sberyozkin@gmail.com>
>>> wrote:
>>>
>>>> Hi Francesco
>>>>
>>>> One thing I can point to is this code:
>>>>
>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>> earch/src/main/java/org/apache/cxf/jaxrs/ext/search/jpa/Abst
>>>> ractJPATypedQueryVisitor.java#L167
>>>>
>>>> There, in the end,
>>>>
>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>> earch/src/main/java/org/apache/cxf/jaxrs/ext/search/jpa/Abst
>>>> ractJPATypedQueryVisitor.java#L181
>>>>
>>>> it branches to either doBuildPredicate() (==> similar to Syncope
>>>> SearchCondVisitor.visitPrimitive) or doBuildCollectionPredicate()
>>>>
>>>> When we have "a.b.c" then if 'b' is a  collection then it would 
>>>> branch to
>>>> doBuildCollectionPredicate.
>>>>
>>>> It was awhile back since I played with the typed JPA2 code, Native 
>>>> one is
>>>> a mystery...
>>>>
>>>> I agree supporting such queries is not easy...but supporting then can
>>>> offer an ultimate search experience :-)
>>>>
>>>> Cheers, Sergey
>>>>
>>>> On 28/06/17 10:26, Francesco Chicchiriccò wrote:
>>>>
>>>>> On 28/06/2017 10:59, Sergey Beryozkin wrote:
>>>>>
>>>>>> Hi Francesco
>>>>>>
>>>>>> Thanks for the explanation.
>>>>>>
>>>>>> I see why the example I pointed to won't be applicable to Syncope.
>>>>>> In that case when the linked beans are available, CXF
>>>>>> AbstractSearchParser will prepare a bean tree which would be 
>>>>>> initialized
>>>>>> with the values from the expression like "a.b.c" and the OOB 
>>>>>> visitor like
>>>>>> JPA2 one takes care of dealing with these linked beans.
>>>>>>
>>>>>> In the SearchBean case it is up to the custom visitor whether to

>>>>>> react
>>>>>> to the '.'s or not where a '.' indicates that for ex 'a' needs to

>>>>>> have 'b'
>>>>>> with a property 'c'.
>>>>>>
>>>>>> Do you reckon Syncope custom visitors can be updated to support such
>>>>>> queries ? Not sure about ElasticSearch but for SQL it should 
>>>>>> probably be
>>>>>> possible...
>>>>>>
>>>>>
>>>>> Maybe in principle yes, it could be possible to support such 
>>>>> queries but:
>>>>>
>>>>> 1. implementation would be rather complex as the query logic is 
>>>>> already
>>>>> quite involved
>>>>> 2. we haven't had may requests for such complex queries so far
>>>>>
>>>>> ...anyway, as you know, volunteers are welcome :-)
>>>>>
>>>>> Regards.
>>>>>
>>>>> On 28/06/17 09:46, Francesco Chicchiriccò wrote:
>>>>>>
>>>>>>> On 28/06/2017 10:41, Sergey Beryozkin wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> I think something similar works for a CXF FIQL JPA2 visitor,
for
>>>>>>>> example:
>>>>>>>>
>>>>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>>>>> earch/src/test/java/org/apache/cxf/jaxrs/ext/search/jpa/JPAT
>>>>>>>> ypedQueryVisitorFiqlTest.java#L65
>>>>>>>>
>>>>>>>> (find the books which have been revied done by Ted)
>>>>>>>>
>>>>>>>
>>>>>>> Hi Sergey,
>>>>>>> that would work if we had straight beans as in the linked sample.
>>>>>>>
>>>>>>> Syncope data model is instead much more involved as new schema
for
>>>>>>> attributes can be defined at runtime: this is the reason why
we 
>>>>>>> have
>>>>>>> SearchCondVisitor [1] translating FIQL into our internal search

>>>>>>> conditions,
>>>>>>> which serve as input to one of available implementations of 
>>>>>>> AnySearchDAO
>>>>>>> like as the default one based on SQL views [2] and another 
>>>>>>> relying on
>>>>>>> Elasticsearch-based [3].
>>>>>>>
>>>>>>> Regards.
>>>>>>>
>>>>>>> [1] https://github.com/apache/syncope/blob/2_0_X/core/persistenc
>>>>>>> e-api/src/main/java/org/apache/syncope/core/persistenc
>>>>>>> e/api/search/SearchCondVisitor.java
>>>>>>> [2] https://github.com/apache/syncope/blob/2_0_X/core/persistenc
>>>>>>> e-jpa/src/main/java/org/apache/syncope/core/persistenc
>>>>>>> e/jpa/dao/JPAAnySearchDAO.java
>>>>>>> [3] https://github.com/apache/syncope/blob/2_0_X/ext/elasticsear
>>>>>>> ch/persistence-jpa/src/main/java/org/apache/syncope/core/
>>>>>>> persistence/jpa/dao/ElasticsearchAnySearchDAO.java
>>>>>>>
>>>>>>> On 28/06/17 08:54, Francesco Chicchiriccò wrote:
>>>>>>>>
>>>>>>>>> On 27/06/2017 18:18, Colm O hEigeartaigh wrote:
>>>>>>>>>
>>>>>>>>>> Thanks Francesco! On a related note, let's say I
have some 
>>>>>>>>>> AnyObjects
>>>>>>>>>> (Printer) with a relationship to other AnyObjects

>>>>>>>>>> (Cartridge). Now I
>>>>>>>>>> want
>>>>>>>>>> to search for a Printer which has a relationship
with a 
>>>>>>>>>> Cartridge
>>>>>>>>>> with a
>>>>>>>>>> "colour" attribute of "blue". Is there a way to do
this via a 
>>>>>>>>>> FIQL
>>>>>>>>>> expression?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> No, you cannot express such condition ATM; you could
do for 
>>>>>>>>> example:
>>>>>>>>>
>>>>>>>>> SyncopeClient.getAnyObjectSearchConditionBuilder("PRINTER").
>>>>>>>>> inRelationships("ce75249b-76e4-44b6-88ae-0841846faceb").
>>>>>>>>> and().is("colour").equalTo("blue").query();
>>>>>>>>>
>>>>>>>>> which translates to FIQL
>>>>>>>>>
>>>>>>>>> $type==PRINTER;$relationships==ce75249b-76e4-44b6-88ae-0841846faceb;colour==blue

>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> but this would rather search for blue printers having
a 
>>>>>>>>> relationship
>>>>>>>>> with an any object with key 
>>>>>>>>> 'ce75249b-76e4-44b6-88ae-0841846faceb'.
>>>>>>>>>
>>>>>>>>> or alternatively
>>>>>>>>>
>>>>>>>>> SyncopeClient.getAnyObjectSearchConditionBuilder("PRINTER").
>>>>>>>>> inRelationshipTypes("WITH_CARTDRIGE").
>>>>>>>>> and().is("color").equalTo("blue").query();
>>>>>>>>>
>>>>>>>>> which translates to FIQL
>>>>>>>>>
>>>>>>>>> $type==PRINTER;$relationshipTypes==WITH_CARTDRIGE;color==blue
>>>>>>>>>
>>>>>>>>> but this would rather search for blue printers having
a 
>>>>>>>>> relationship
>>>>>>>>> on type  WITH_CARTDRIGE.
>>>>>>>>>
>>>>>>>>> Regards.
>>>>>>>>>
>>>>>>>>> On Tue, Jun 27, 2017 at 4:29 PM, Francesco Chicchiriccò
<
>>>>>>>>>> ilgrosso@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>> On 27/06/2017 17:24, Colm O hEigeartaigh wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>>>
>>>>>>>>>>>> How can I retrieve a list of AnyObjects?
The following 
>>>>>>>>>>>> returns a
>>>>>>>>>>>> 400:
>>>>>>>>>>>>
>>>>>>>>>>>> curl -I -X GET -u admin:password
>>>>>>>>>>>> http://localhost:9080/syncope/rest/anyObjects
>>>>>>>>>>>>
>>>>>>>>>>>> You must at least provide the AnyType, e.g.
>>>>>>>>>>>
>>>>>>>>>>> http://localhost:9080/syncope/rest/anyObjects;fiql=%24type%3
>>>>>>>>>>> D%3DPRINTER
>>>>>>>>>>>
>>>>>>>>>>> Regards. 

-- 
Francesco Chicchiriccò

Tirasa - Open Source Excellence
http://www.tirasa.net/

Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/


Mime
View raw message