syncope-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Francesco Chicchiriccò <ilgro...@apache.org>
Subject Re: AnyObjects query
Date Mon, 03 Jul 2017 07:05:54 GMT
On 30/06/2017 18:23, Colm O hEigeartaigh wrote:
> Hi Francesco,
>
> On Fri, Jun 30, 2017 at 3:42 PM, Francesco Chicchiriccò <ilgrosso@apache.org>
wrote:
>> At persistence level, Relationships have leftEnd and rightEnd:
>>
>> https://github.com/apache/syncope/blob/2_0_X/core/persistence-api/src/main/java/org/apache/syncope/core/persistence/api/
>> entity/Relationship.java
>>
>> However, since Relationships can only be read via REST through one of
>> their ends (User <-> AnyObject or AnyObject <-> AnyObject), then it would
>> make more sense to always report the other end via RelationshipTO, rather
>> than reporting the right end, as doing currently:
>>
>> https://github.com/apache/syncope/blob/2_0_X/common/lib/src/main/java/org/apache/syncope/common/lib/to/RelationshipTO.java
>>
>> I would say the current implementation is anyway not the best, and worths
>> an improvement, maybe for 2.0.5 as there are ATM 0 issues open for 2.0.4
>> and CXF 3.1.12 was just released - meaning that we could start our own
>> release process on Monday.
> OK great! I will create a JIRA for this for 2.0.5. One other related query
> - let's say I get back my list of blue cartridges who are in a relationship
> with a printer. I'll be able to extract (I guess) the leftKey from the
> relationships, so I'll know the "key" of the printers. Now how do I
> retrieve all printers that match any of the given keys? Looking at
> SpecialAttr there is not a "$key" available, or is there another way of
> search for this?

The key is not a special attribute, so it would simply be "key", not 
"$key" in FIQL.

FYI all the special attributes are

https://github.com/apache/syncope/blob/2_0_X/common/lib/src/main/java/org/apache/syncope/common/lib/search/SpecialAttr.java

Regards.

> On Thu, Jun 29, 2017 at 10:19 AM, Francesco Chicchiriccò <
>>> ilgrosso@apache.org> wrote:
>>>
>>> On 29/06/2017 11:04, Sergey Beryozkin wrote:
>>>> Hi Francesco
>>>>> OK. So, as far as the Syncope returning the list of printers to the
>>>>> client code issuing the query, how would it guess that when someone
>>>>> searches for the blue cartridges it is actually a list of printers is
>>>>> expected ? Where will this code which next goes over the list of all
the
>>>>> cartridges and prepare a list of Printers will sit ?
>>>>>
>>>>> As far as the demand is concerned: it is driven by a too long URI
>>>>> production issue Colm referred to
>>>>>
>>>>> The URI too long is generated if you do the search the way Colm
>>>> describes
>>>> it.
>>>>
>>>> If instead, by following Fabio's suggestion, you search for all
>>>> Cartdriges
>>>> with "colour==blue" having a certain RelationshipType (which you have
>>>> prior
>>>> defined, with Printers, say it is named 'PRINTER_CARTDRIGE_ASSIGNMENT'),
>>>> then you need a single query with the following FIQL:
>>>>
>>>> $type==CARTDRIGE;color==blue;$relationshipTypes==PRINTER_CAR
>>>> TDRIGE_ASSIGNMENT
>>>>
>>>> Such query will return a list of AnyObjectTO instances, each of which
>>>> containing a list of RelationshipTO, where each RelationshipTO instance
>>>> has
>>>> a type field.
>>>>
>>>> Hence, to get "the printers with a blue cartridge", it will be enough to
>>>> iterate over the returned list of AnyObjectTO instances (of type
>>>> CARTDRIGE), find the RelationshipTO instances of type
>>>> PRINTER_CARTDRIGE_ASSIGNMENT, and finally get the other side of the
>>>> relationship, e.g. the printer.
>>>> Code-wise:
>>>>
>>>>           PagedResult<AnyObjectTO> cartdriges = ...; // returned by
the
>>>> search
>>>>           List<String> printers = new ArrayList<>();
>>>>           for (AnyObjectTO cartdrige: cartdriges.getResult()) {
>>>>               for (RelationshipTO relationship:
>>>> cartdrige.getRelationships()) {
>>>>                   if ("PRINTER_CARTDRIGE_ASSIGNMENT
>>>> ".equals(relationship.getType())) {
>>>>                       printers.add(relationship.getRightKey());
>>>>                   }
>>>>               }
>>>>           }
>>>>
>>>> Anyway, as said, there is always room to improve.
>>>> Regards.
>>>>
>>>>
>>>> On 29/06/17 09:53, Francesco Chicchiriccò wrote:
>>>>
>>>>> On 29/06/2017 10:52, Sergey Beryozkin wrote:
>>>>>> Hi Fabio
>>>>>>> That would not work at the generic level as it would require
a 2-way
>>>>>>> relationship (cartridge - printer for ex) which is not always
>>>>>>> realistic to
>>>>>>> expect,
>>>>>>>
>>>>>>> In the Syncope data model, all Relationships are 2-way.
>>>>>> I agree with Fabio: the use case proposed by Colm is doable with
>>>>>> current
>>>>>> implementation.
>>>>>>
>>>>>> Naturally, there is always room for improvements, but I don't see
much
>>>>>> demand ATM.
>>>>>>
>>>>>> Regards.
>>>>>>
>>>>>> On 29/06/17 09:07, Fabio Martelli wrote:
>>>>>>
>>>>>>>> Il 28/06/2017 18:34, Colm O hEigeartaigh ha scritto:
>>>>>>>>
>>>>>>>> Thanks for the feedback guys! Let me just expand a bit on
the
>>>>>>>>> motivation
>>>>>>>>> behind my previous example....
>>>>>>>>>
>>>>>>>>> Let's say I'm managing hundreds of printers each of which
have a
>>>>>>>>> relationship to a cartridge (of which there are many
hundreds) with
>>>>>>>>> a
>>>>>>>>> colour attribute. I want to find the printers with a
blue cartridge.
>>>>>>>>>
>>>>>>>>> So I first make a search for a list of "blue" cartridges.
Then I
>>>>>>>>> search for
>>>>>>>>> the printers that have a relationship with these cartridges.
The
>>>>>>>>> problem is
>>>>>>>>> on the second search I end up with a ginormous search
expression
>>>>>>>>> "relationships%3D%3D7db4512-ad25-40e8-bc78-63ad25c0e894%2C%
>>>>>>>>> 24relationships%3D%3D16dc6acd-6.....".
>>>>>>>>> " that could be an invalid URL.
>>>>>>>>>
>>>>>>>>> Is there a better way of handling it than this?
>>>>>>>>>
>>>>>>>>> Hi Colm, maybe you can search all the cartridges blue
with a
>>>>>>>> relationship with a printer.
>>>>>>>> The result will be a list of cartridges. Each item will be
reported
>>>>>>>> including a specific field relationships (if I well remember)
about
>>>>>>>> all the
>>>>>>>> printers related to the item.
>>>>>>>> In this case, you will have all the info you need retrieved
by a
>>>>>>>> single short query. Don't you agree?
>>>>>>>>
>>>>>>>> BR,
>>>>>>>> F.
>>>>>>>>
>>>>>>>> Colm.
>>>>>>>>> On Wed, Jun 28, 2017 at 10:52 AM, Sergey Beryozkin <
>>>>>>>>> sberyozkin@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi Francesco
>>>>>>>>>
>>>>>>>>>> One thing I can point to is this code:
>>>>>>>>>>
>>>>>>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>>>>>>> earch/src/main/java/org/apache/cxf/jaxrs/ext/search/jpa/Abst
>>>>>>>>>> ractJPATypedQueryVisitor.java#L167
>>>>>>>>>>
>>>>>>>>>> There, in the end,
>>>>>>>>>>
>>>>>>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>>>>>>> earch/src/main/java/org/apache/cxf/jaxrs/ext/search/jpa/Abst
>>>>>>>>>> ractJPATypedQueryVisitor.java#L181
>>>>>>>>>>
>>>>>>>>>> it branches to either doBuildPredicate() (==>
similar to Syncope
>>>>>>>>>> SearchCondVisitor.visitPrimitive) or doBuildCollectionPredicate()
>>>>>>>>>>
>>>>>>>>>> When we have "a.b.c" then if 'b' is a  collection
then it would
>>>>>>>>>> branch to
>>>>>>>>>> doBuildCollectionPredicate.
>>>>>>>>>>
>>>>>>>>>> It was awhile back since I played with the typed
JPA2 code, Native
>>>>>>>>>> one is
>>>>>>>>>> a mystery...
>>>>>>>>>>
>>>>>>>>>> I agree supporting such queries is not easy...but
supporting then
>>>>>>>>>> can
>>>>>>>>>> offer an ultimate search experience :-)
>>>>>>>>>>
>>>>>>>>>> Cheers, Sergey
>>>>>>>>>>
>>>>>>>>>> On 28/06/17 10:26, Francesco Chicchiriccò wrote:
>>>>>>>>>>
>>>>>>>>>> On 28/06/2017 10:59, Sergey Beryozkin wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Francesco
>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the explanation.
>>>>>>>>>>>>
>>>>>>>>>>>> I see why the example I pointed to won't
be applicable to
>>>>>>>>>>>> Syncope.
>>>>>>>>>>>> In that case when the linked beans are available,
CXF
>>>>>>>>>>>> AbstractSearchParser will prepare a bean
tree which would be
>>>>>>>>>>>> initialized
>>>>>>>>>>>> with the values from the expression like
"a.b.c" and the OOB
>>>>>>>>>>>> visitor like
>>>>>>>>>>>> JPA2 one takes care of dealing with these
linked beans.
>>>>>>>>>>>>
>>>>>>>>>>>> In the SearchBean case it is up to the custom
visitor whether to
>>>>>>>>>>>> react
>>>>>>>>>>>> to the '.'s or not where a '.' indicates
that for ex 'a' needs to
>>>>>>>>>>>> have 'b'
>>>>>>>>>>>> with a property 'c'.
>>>>>>>>>>>>
>>>>>>>>>>>> Do you reckon Syncope custom visitors can
be updated to support
>>>>>>>>>>>> such
>>>>>>>>>>>> queries ? Not sure about ElasticSearch but
for SQL it should
>>>>>>>>>>>> probably be
>>>>>>>>>>>> possible...
>>>>>>>>>>>>
>>>>>>>>>>>> Maybe in principle yes, it could be possible
to support such
>>>>>>>>>>>>
>>>>>>>>>>> queries but:
>>>>>>>>>>>
>>>>>>>>>>> 1. implementation would be rather complex as
the query logic is
>>>>>>>>>>> already
>>>>>>>>>>> quite involved
>>>>>>>>>>> 2. we haven't had may requests for such complex
queries so far
>>>>>>>>>>>
>>>>>>>>>>> ...anyway, as you know, volunteers are welcome
:-)
>>>>>>>>>>>
>>>>>>>>>>> Regards.
>>>>>>>>>>>
>>>>>>>>>>> On 28/06/17 09:46, Francesco Chicchiriccò wrote:
>>>>>>>>>>>
>>>>>>>>>>> On 28/06/2017 10:41, Sergey Beryozkin wrote:
>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think something similar works for
a CXF FIQL JPA2 visitor,
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>> example:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>>>>>>>>>>> earch/src/test/java/org/apache/cxf/jaxrs/ext/search/jpa/JPAT
>>>>>>>>>>>>>> ypedQueryVisitorFiqlTest.java#L65
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (find the books which have been revied
done by Ted)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Sergey,
>>>>>>>>>>>>>>
>>>>>>>>>>>>> that would work if we had straight beans
as in the linked
>>>>>>>>>>>>> sample.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Syncope data model is instead much more
involved as new schema
>>>>>>>>>>>>> for
>>>>>>>>>>>>> attributes can be defined at runtime:
this is the reason why we
>>>>>>>>>>>>> have
>>>>>>>>>>>>> SearchCondVisitor [1] translating FIQL
into our internal search
>>>>>>>>>>>>> conditions,
>>>>>>>>>>>>> which serve as input to one of available
implementations of
>>>>>>>>>>>>> AnySearchDAO
>>>>>>>>>>>>> like as the default one based on SQL
views [2] and another
>>>>>>>>>>>>> relying on
>>>>>>>>>>>>> Elasticsearch-based [3].
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] https://github.com/apache/sync
>>>>>>>>>>>>> ope/blob/2_0_X/core/persistenc
>>>>>>>>>>>>> e-api/src/main/java/org/apache/syncope/core/persistenc
>>>>>>>>>>>>> e/api/search/SearchCondVisitor.java
>>>>>>>>>>>>> [2] https://github.com/apache/sync
>>>>>>>>>>>>> ope/blob/2_0_X/core/persistenc
>>>>>>>>>>>>> e-jpa/src/main/java/org/apache/syncope/core/persistenc
>>>>>>>>>>>>> e/jpa/dao/JPAAnySearchDAO.java
>>>>>>>>>>>>> [3] https://github.com/apache/sync
>>>>>>>>>>>>> ope/blob/2_0_X/ext/elasticsear
>>>>>>>>>>>>> ch/persistence-jpa/src/main/java/org/apache/syncope/core/
>>>>>>>>>>>>> persistence/jpa/dao/ElasticsearchAnySearchDAO.java
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 28/06/17 08:54, Francesco Chicchiriccò
wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 27/06/2017 18:18, Colm O hEigeartaigh
wrote:
>>>>>>>>>>>>>>> Thanks Francesco! On a related
note, let's say I have some
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> AnyObjects
>>>>>>>>>>>>>>>> (Printer) with a relationship
to other AnyObjects
>>>>>>>>>>>>>>>> (Cartridge).
>>>>>>>>>>>>>>>> Now I
>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>> to search for a Printer which
has a relationship with a
>>>>>>>>>>>>>>>> Cartridge
>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>> "colour" attribute of "blue".
Is there a way to do this via a
>>>>>>>>>>>>>>>> FIQL
>>>>>>>>>>>>>>>> expression?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> No, you cannot express such
condition ATM; you could do for
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> example:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> SyncopeClient.getAnyObjectSearchConditionBuilder("PRINTER").
>>>>>>>>>>>>>>> inRelationships("ce75249b-76e4-44b6-88ae-0841846faceb").
>>>>>>>>>>>>>>> and().is("colour").equalTo("blue").query();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> which translates to FIQL
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> $type==PRINTER;$relationships==ce75249b-76e4-44b6-88ae-08418
>>>>>>>>>>>>>>> 46faceb;colour==blue
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> but this would rather search
for blue printers having a
>>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>> with an any object with key 'ce75249b-76e4-44b6-88ae-08418
>>>>>>>>>>>>>>> 46faceb'.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> or alternatively
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> SyncopeClient.getAnyObjectSearchConditionBuilder("PRINTER").
>>>>>>>>>>>>>>> inRelationshipTypes("WITH_CARTDRIGE").
>>>>>>>>>>>>>>> and().is("color").equalTo("blue").query();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> which translates to FIQL
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> $type==PRINTER;$relationshipTypes==WITH_CARTDRIGE;color==blue
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> but this would rather search
for blue printers having a
>>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>> on type  WITH_CARTDRIGE.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Jun 27, 2017 at 4:29
PM, Francesco Chicchiriccò <
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ilgrosso@apache.org> wrote:
>>>>>>>>>>>>>>>> On 27/06/2017 17:24, Colm
O hEigeartaigh wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>> How can I retrieve a
list of AnyObjects? The following
>>>>>>>>>>>>>>>>>> returns a
>>>>>>>>>>>>>>>>>> 400:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> curl -I -X GET -u
admin:password
>>>>>>>>>>>>>>>>>> http://localhost:9080/syncope/rest/anyObjects
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> You must at least
provide the AnyType, e.g.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> http://localhost:9080/syncope/
>>>>>>>>>>>>>>>>> rest/anyObjects;fiql=%24type%3
>>>>>>>>>>>>>>>>> D%3DPRINTER
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>
>> --
>> Francesco Chicchiriccò
>>
>> Tirasa - Open Source Excellence
>> http://www.tirasa.net/
>>
>> Member at The Apache Software Foundation
>> Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
>> http://home.apache.org/~ilgrosso/
>>
>>
>

-- 
Francesco Chicchiriccò

Tirasa - Open Source Excellence
http://www.tirasa.net/

Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/


Mime
View raw message