syncope-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Beryozkin <sberyoz...@gmail.com>
Subject Re: AnyObjects query
Date Tue, 04 Jul 2017 14:11:00 GMT
Hi Francesco

Sure, I agree. I just wanted to send an update on my own basic 
experiment involving no typed data but assuming there are many tables 
available :-)

May be we have a new GSOC idea here :-)

Thanks, Sergey

On 04/07/17 15:04, Francesco Chicchiriccò wrote:
> On 04/07/2017 15:57, Sergey Beryozkin wrote:
>> I did some experiments in CXF:
>>
>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/search/src/test/java/org/apache/cxf/jaxrs/ext/search/sql/SQLHierarchicalQueryTest.java

>>
>>
>> Indeed, making it work at the generic level, without the typed model, 
>> requires some flexibility at the code level :-), with the proper 
>> customization supported if needed.
>>
>> For example, in the CXF code, the name of the joining key is deduced 
>> at the moment, given that the primary table is 'printers' and the 
>> joining one is 'cartridges' then it is assumed that a 'printers' table 
>> has an 'id' primary key while 'cartridges' has a 'printer_id' foreign 
>> key ('printers' minus 's' + "_id").
>>
>> I think the custom visitor optimized around the specific data model 
>> can do it much better :-)
> 
> Hi Sergey,
> thanks for your experiments.
> 
> Please consider that we don't even have two distinct tables 'printers' 
> and 'cartridges' but a single "AnyObjects" table, where attributes and 
> their values are stored into separate tables, and the actual search is 
> performed against the views
> 
> https://github.com/apache/syncope/blob/2_0_X/core/persistence-jpa/src/main/resources/views.xml#L129-L199

> 
> 
> depending on the actual arguments of the requested search.
> 
> A said, enhancing our custom visitor (e.g. 
> org.apache.syncope.core.persistence.api.search.SearchCondVisitor) to 
> cope with such searches is definitely not a trivial task.
> 
> Regards.
> 
>> On 29/06/17 09:49, Sergey Beryozkin wrote:
>>> Yes, as far as the convention is concerned, one would express it
>>> as
>>>
>>> GET
>>>
>>> /printers?_s=cartridges.colour=blue
>>>
>>> then at the the next stage it depends if a typed model already 
>>> exists, if yes, then it can work OOB with for ex JPA2 CXF visitor.
>>> In case of Syncope the model is dynamic, hence the custom Syncope 
>>> visitor deals with a string such 'cartridges.colour=blue' itself but 
>>> at the moment it does not attach any significance to a '.'.
>>>
>>> It will need to be enhanced for it to process '.' and build a more 
>>> sophisticated Natve JPA query. It is doable, I agree with Francesco 
>>> it will be more involved...
>>>
>>> I propose that at least we create a JIRA to track the enhancement 
>>> request.
>>>
>>> I can experiment at the CXF level to enhance its SQLPrinterVisitor to 
>>> see what sort of processing can be required, I'm certain it can be done
>>>
>>> Sergey
>>>
>>> On 28/06/17 17:34, Colm O hEigeartaigh wrote:
>>>> Thanks for the feedback guys! Let me just expand a bit on the 
>>>> motivation
>>>> behind my previous example....
>>>>
>>>> Let's say I'm managing hundreds of printers each of which have a
>>>> relationship to a cartridge (of which there are many hundreds) with a
>>>> colour attribute. I want to find the printers with a blue cartridge.
>>>>
>>>> So I first make a search for a list of "blue" cartridges. Then I 
>>>> search for
>>>> the printers that have a relationship with these cartridges. The 
>>>> problem is
>>>> on the second search I end up with a ginormous search expression
>>>> "relationships%3D%3D7db4512-ad25-40e8-bc78-63ad25c0e894%2C%24relationships%3D%3D16dc6acd-6.....".

>>>>
>>>> " that could be an invalid URL.
>>>>
>>>> Is there a better way of handling it than this?
>>>>
>>>> Colm.
>>>>
>>>> On Wed, Jun 28, 2017 at 10:52 AM, Sergey Beryozkin 
>>>> <sberyozkin@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Francesco
>>>>>
>>>>> One thing I can point to is this code:
>>>>>
>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>> earch/src/main/java/org/apache/cxf/jaxrs/ext/search/jpa/Abst
>>>>> ractJPATypedQueryVisitor.java#L167
>>>>>
>>>>> There, in the end,
>>>>>
>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>> earch/src/main/java/org/apache/cxf/jaxrs/ext/search/jpa/Abst
>>>>> ractJPATypedQueryVisitor.java#L181
>>>>>
>>>>> it branches to either doBuildPredicate() (==> similar to Syncope
>>>>> SearchCondVisitor.visitPrimitive) or doBuildCollectionPredicate()
>>>>>
>>>>> When we have "a.b.c" then if 'b' is a  collection then it would 
>>>>> branch to
>>>>> doBuildCollectionPredicate.
>>>>>
>>>>> It was awhile back since I played with the typed JPA2 code, Native 
>>>>> one is
>>>>> a mystery...
>>>>>
>>>>> I agree supporting such queries is not easy...but supporting then can
>>>>> offer an ultimate search experience :-)
>>>>>
>>>>> Cheers, Sergey
>>>>>
>>>>> On 28/06/17 10:26, Francesco Chicchiriccò wrote:
>>>>>
>>>>>> On 28/06/2017 10:59, Sergey Beryozkin wrote:
>>>>>>
>>>>>>> Hi Francesco
>>>>>>>
>>>>>>> Thanks for the explanation.
>>>>>>>
>>>>>>> I see why the example I pointed to won't be applicable to Syncope.
>>>>>>> In that case when the linked beans are available, CXF
>>>>>>> AbstractSearchParser will prepare a bean tree which would be

>>>>>>> initialized
>>>>>>> with the values from the expression like "a.b.c" and the OOB

>>>>>>> visitor like
>>>>>>> JPA2 one takes care of dealing with these linked beans.
>>>>>>>
>>>>>>> In the SearchBean case it is up to the custom visitor whether
to 
>>>>>>> react
>>>>>>> to the '.'s or not where a '.' indicates that for ex 'a' needs
to 
>>>>>>> have 'b'
>>>>>>> with a property 'c'.
>>>>>>>
>>>>>>> Do you reckon Syncope custom visitors can be updated to support
such
>>>>>>> queries ? Not sure about ElasticSearch but for SQL it should

>>>>>>> probably be
>>>>>>> possible...
>>>>>>>
>>>>>>
>>>>>> Maybe in principle yes, it could be possible to support such 
>>>>>> queries but:
>>>>>>
>>>>>> 1. implementation would be rather complex as the query logic is 
>>>>>> already
>>>>>> quite involved
>>>>>> 2. we haven't had may requests for such complex queries so far
>>>>>>
>>>>>> ...anyway, as you know, volunteers are welcome :-)
>>>>>>
>>>>>> Regards.
>>>>>>
>>>>>> On 28/06/17 09:46, Francesco Chicchiriccò wrote:
>>>>>>>
>>>>>>>> On 28/06/2017 10:41, Sergey Beryozkin wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> I think something similar works for a CXF FIQL JPA2 visitor,
for
>>>>>>>>> example:
>>>>>>>>>
>>>>>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>>>>>> earch/src/test/java/org/apache/cxf/jaxrs/ext/search/jpa/JPAT
>>>>>>>>> ypedQueryVisitorFiqlTest.java#L65
>>>>>>>>>
>>>>>>>>> (find the books which have been revied done by Ted)
>>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Sergey,
>>>>>>>> that would work if we had straight beans as in the linked
sample.
>>>>>>>>
>>>>>>>> Syncope data model is instead much more involved as new schema
for
>>>>>>>> attributes can be defined at runtime: this is the reason
why we 
>>>>>>>> have
>>>>>>>> SearchCondVisitor [1] translating FIQL into our internal
search 
>>>>>>>> conditions,
>>>>>>>> which serve as input to one of available implementations
of 
>>>>>>>> AnySearchDAO
>>>>>>>> like as the default one based on SQL views [2] and another

>>>>>>>> relying on
>>>>>>>> Elasticsearch-based [3].
>>>>>>>>
>>>>>>>> Regards.
>>>>>>>>
>>>>>>>> [1] https://github.com/apache/syncope/blob/2_0_X/core/persistenc
>>>>>>>> e-api/src/main/java/org/apache/syncope/core/persistenc
>>>>>>>> e/api/search/SearchCondVisitor.java
>>>>>>>> [2] https://github.com/apache/syncope/blob/2_0_X/core/persistenc
>>>>>>>> e-jpa/src/main/java/org/apache/syncope/core/persistenc
>>>>>>>> e/jpa/dao/JPAAnySearchDAO.java
>>>>>>>> [3] https://github.com/apache/syncope/blob/2_0_X/ext/elasticsear
>>>>>>>> ch/persistence-jpa/src/main/java/org/apache/syncope/core/
>>>>>>>> persistence/jpa/dao/ElasticsearchAnySearchDAO.java
>>>>>>>>
>>>>>>>> On 28/06/17 08:54, Francesco Chicchiriccò wrote:
>>>>>>>>>
>>>>>>>>>> On 27/06/2017 18:18, Colm O hEigeartaigh wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks Francesco! On a related note, let's say
I have some 
>>>>>>>>>>> AnyObjects
>>>>>>>>>>> (Printer) with a relationship to other AnyObjects

>>>>>>>>>>> (Cartridge). Now I
>>>>>>>>>>> want
>>>>>>>>>>> to search for a Printer which has a relationship
with a 
>>>>>>>>>>> Cartridge
>>>>>>>>>>> with a
>>>>>>>>>>> "colour" attribute of "blue". Is there a way
to do this via a 
>>>>>>>>>>> FIQL
>>>>>>>>>>> expression?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> No, you cannot express such condition ATM; you could
do for 
>>>>>>>>>> example:
>>>>>>>>>>
>>>>>>>>>> SyncopeClient.getAnyObjectSearchConditionBuilder("PRINTER").
>>>>>>>>>> inRelationships("ce75249b-76e4-44b6-88ae-0841846faceb").
>>>>>>>>>> and().is("colour").equalTo("blue").query();
>>>>>>>>>>
>>>>>>>>>> which translates to FIQL
>>>>>>>>>>
>>>>>>>>>> $type==PRINTER;$relationships==ce75249b-76e4-44b6-88ae-0841846faceb;colour==blue

>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> but this would rather search for blue printers having
a 
>>>>>>>>>> relationship
>>>>>>>>>> with an any object with key 
>>>>>>>>>> 'ce75249b-76e4-44b6-88ae-0841846faceb'.
>>>>>>>>>>
>>>>>>>>>> or alternatively
>>>>>>>>>>
>>>>>>>>>> SyncopeClient.getAnyObjectSearchConditionBuilder("PRINTER").
>>>>>>>>>> inRelationshipTypes("WITH_CARTDRIGE").
>>>>>>>>>> and().is("color").equalTo("blue").query();
>>>>>>>>>>
>>>>>>>>>> which translates to FIQL
>>>>>>>>>>
>>>>>>>>>> $type==PRINTER;$relationshipTypes==WITH_CARTDRIGE;color==blue
>>>>>>>>>>
>>>>>>>>>> but this would rather search for blue printers having
a 
>>>>>>>>>> relationship
>>>>>>>>>> on type  WITH_CARTDRIGE.
>>>>>>>>>>
>>>>>>>>>> Regards.
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 27, 2017 at 4:29 PM, Francesco Chicchiriccò
<
>>>>>>>>>>> ilgrosso@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>> On 27/06/2017 17:24, Colm O hEigeartaigh wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>
>>>>>>>>>>>>> How can I retrieve a list of AnyObjects?
The following 
>>>>>>>>>>>>> returns a
>>>>>>>>>>>>> 400:
>>>>>>>>>>>>>
>>>>>>>>>>>>> curl -I -X GET -u admin:password
>>>>>>>>>>>>> http://localhost:9080/syncope/rest/anyObjects
>>>>>>>>>>>>>
>>>>>>>>>>>>> You must at least provide the AnyType,
e.g.
>>>>>>>>>>>>
>>>>>>>>>>>> http://localhost:9080/syncope/rest/anyObjects;fiql=%24type%3
>>>>>>>>>>>> D%3DPRINTER
>>>>>>>>>>>>
>>>>>>>>>>>> Regards. 
> 

Mime
View raw message