syncope-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colm O hEigeartaigh <cohei...@apache.org>
Subject Re: AnyObjects query
Date Fri, 30 Jun 2017 16:23:13 GMT
Hi Francesco,


On Fri, Jun 30, 2017 at 3:42 PM, Francesco Chicchiriccò <ilgrosso@apache.org
> wrote:


> At persistence level, Relationships have leftEnd and rightEnd:
>
> https://github.com/apache/syncope/blob/2_0_X/core/persistenc
> e-api/src/main/java/org/apache/syncope/core/persistence/api/
> entity/Relationship.java
>
> However, since Relationships can only be read via REST through one of
> their ends (User <-> AnyObject or AnyObject <-> AnyObject), then it would
> make more sense to always report the other end via RelationshipTO, rather
> than reporting the right end, as doing currently:
>
> https://github.com/apache/syncope/blob/2_0_X/common/lib/src/
> main/java/org/apache/syncope/common/lib/to/RelationshipTO.java
>
> I would say the current implementation is anyway not the best, and worths
> an improvement, maybe for 2.0.5 as there are ATM 0 issues open for 2.0.4
> and CXF 3.1.12 was just released - meaning that we could start our own
> release process on Monday.
>

OK great! I will create a JIRA for this for 2.0.5. One other related query
- let's say I get back my list of blue cartridges who are in a relationship
with a printer. I'll be able to extract (I guess) the leftKey from the
relationships, so I'll know the "key" of the printers. Now how do I
retrieve all printers that match any of the given keys? Looking at
SpecialAttr there is not a "$key" available, or is there another way of
search for this?

Colm.



>
> Regards.
>
>
> On Thu, Jun 29, 2017 at 10:19 AM, Francesco Chicchiriccò <
>> ilgrosso@apache.org> wrote:
>>
>> On 29/06/2017 11:04, Sergey Beryozkin wrote:
>>>
>>> Hi Francesco
>>>>
>>>> OK. So, as far as the Syncope returning the list of printers to the
>>>> client code issuing the query, how would it guess that when someone
>>>> searches for the blue cartridges it is actually a list of printers is
>>>> expected ? Where will this code which next goes over the list of all the
>>>> cartridges and prepare a list of Printers will sit ?
>>>>
>>>> As far as the demand is concerned: it is driven by a too long URI
>>>> production issue Colm referred to
>>>>
>>>> The URI too long is generated if you do the search the way Colm
>>> describes
>>> it.
>>>
>>> If instead, by following Fabio's suggestion, you search for all
>>> Cartdriges
>>> with "colour==blue" having a certain RelationshipType (which you have
>>> prior
>>> defined, with Printers, say it is named 'PRINTER_CARTDRIGE_ASSIGNMENT'),
>>> then you need a single query with the following FIQL:
>>>
>>> $type==CARTDRIGE;color==blue;$relationshipTypes==PRINTER_CAR
>>> TDRIGE_ASSIGNMENT
>>>
>>> Such query will return a list of AnyObjectTO instances, each of which
>>> containing a list of RelationshipTO, where each RelationshipTO instance
>>> has
>>> a type field.
>>>
>>> Hence, to get "the printers with a blue cartridge", it will be enough to
>>> iterate over the returned list of AnyObjectTO instances (of type
>>> CARTDRIGE), find the RelationshipTO instances of type
>>> PRINTER_CARTDRIGE_ASSIGNMENT, and finally get the other side of the
>>> relationship, e.g. the printer.
>>> Code-wise:
>>>
>>>          PagedResult<AnyObjectTO> cartdriges = ...; // returned by the
>>> search
>>>          List<String> printers = new ArrayList<>();
>>>          for (AnyObjectTO cartdrige: cartdriges.getResult()) {
>>>              for (RelationshipTO relationship:
>>> cartdrige.getRelationships()) {
>>>                  if ("PRINTER_CARTDRIGE_ASSIGNMENT
>>> ".equals(relationship.getType())) {
>>>                      printers.add(relationship.getRightKey());
>>>                  }
>>>              }
>>>          }
>>>
>>> Anyway, as said, there is always room to improve.
>>> Regards.
>>>
>>>
>>> On 29/06/17 09:53, Francesco Chicchiriccò wrote:
>>>
>>>> On 29/06/2017 10:52, Sergey Beryozkin wrote:
>>>>>
>>>>> Hi Fabio
>>>>>>
>>>>>> That would not work at the generic level as it would require a 2-way
>>>>>> relationship (cartridge - printer for ex) which is not always
>>>>>> realistic to
>>>>>> expect,
>>>>>>
>>>>>> In the Syncope data model, all Relationships are 2-way.
>>>>>
>>>>> I agree with Fabio: the use case proposed by Colm is doable with
>>>>> current
>>>>> implementation.
>>>>>
>>>>> Naturally, there is always room for improvements, but I don't see much
>>>>> demand ATM.
>>>>>
>>>>> Regards.
>>>>>
>>>>> On 29/06/17 09:07, Fabio Martelli wrote:
>>>>>
>>>>>>
>>>>>>> Il 28/06/2017 18:34, Colm O hEigeartaigh ha scritto:
>>>>>>>
>>>>>>> Thanks for the feedback guys! Let me just expand a bit on the
>>>>>>>> motivation
>>>>>>>> behind my previous example....
>>>>>>>>
>>>>>>>> Let's say I'm managing hundreds of printers each of which
have a
>>>>>>>> relationship to a cartridge (of which there are many hundreds)
with
>>>>>>>> a
>>>>>>>> colour attribute. I want to find the printers with a blue
cartridge.
>>>>>>>>
>>>>>>>> So I first make a search for a list of "blue" cartridges.
Then I
>>>>>>>> search for
>>>>>>>> the printers that have a relationship with these cartridges.
The
>>>>>>>> problem is
>>>>>>>> on the second search I end up with a ginormous search expression
>>>>>>>> "relationships%3D%3D7db4512-ad25-40e8-bc78-63ad25c0e894%2C%
>>>>>>>> 24relationships%3D%3D16dc6acd-6.....".
>>>>>>>> " that could be an invalid URL.
>>>>>>>>
>>>>>>>> Is there a better way of handling it than this?
>>>>>>>>
>>>>>>>> Hi Colm, maybe you can search all the cartridges blue with
a
>>>>>>> relationship with a printer.
>>>>>>> The result will be a list of cartridges. Each item will be reported
>>>>>>> including a specific field relationships (if I well remember)
about
>>>>>>> all the
>>>>>>> printers related to the item.
>>>>>>> In this case, you will have all the info you need retrieved by
a
>>>>>>> single short query. Don't you agree?
>>>>>>>
>>>>>>> BR,
>>>>>>> F.
>>>>>>>
>>>>>>> Colm.
>>>>>>>>
>>>>>>>> On Wed, Jun 28, 2017 at 10:52 AM, Sergey Beryozkin <
>>>>>>>> sberyozkin@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi Francesco
>>>>>>>>
>>>>>>>>> One thing I can point to is this code:
>>>>>>>>>
>>>>>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>>>>>> earch/src/main/java/org/apache/cxf/jaxrs/ext/search/jpa/Abst
>>>>>>>>> ractJPATypedQueryVisitor.java#L167
>>>>>>>>>
>>>>>>>>> There, in the end,
>>>>>>>>>
>>>>>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>>>>>> earch/src/main/java/org/apache/cxf/jaxrs/ext/search/jpa/Abst
>>>>>>>>> ractJPATypedQueryVisitor.java#L181
>>>>>>>>>
>>>>>>>>> it branches to either doBuildPredicate() (==> similar
to Syncope
>>>>>>>>> SearchCondVisitor.visitPrimitive) or doBuildCollectionPredicate()
>>>>>>>>>
>>>>>>>>> When we have "a.b.c" then if 'b' is a  collection then
it would
>>>>>>>>> branch to
>>>>>>>>> doBuildCollectionPredicate.
>>>>>>>>>
>>>>>>>>> It was awhile back since I played with the typed JPA2
code, Native
>>>>>>>>> one is
>>>>>>>>> a mystery...
>>>>>>>>>
>>>>>>>>> I agree supporting such queries is not easy...but supporting
then
>>>>>>>>> can
>>>>>>>>> offer an ultimate search experience :-)
>>>>>>>>>
>>>>>>>>> Cheers, Sergey
>>>>>>>>>
>>>>>>>>> On 28/06/17 10:26, Francesco Chicchiriccò wrote:
>>>>>>>>>
>>>>>>>>> On 28/06/2017 10:59, Sergey Beryozkin wrote:
>>>>>>>>>
>>>>>>>>>> Hi Francesco
>>>>>>>>>>
>>>>>>>>>>> Thanks for the explanation.
>>>>>>>>>>>
>>>>>>>>>>> I see why the example I pointed to won't be applicable
to
>>>>>>>>>>> Syncope.
>>>>>>>>>>> In that case when the linked beans are available,
CXF
>>>>>>>>>>> AbstractSearchParser will prepare a bean tree
which would be
>>>>>>>>>>> initialized
>>>>>>>>>>> with the values from the expression like "a.b.c"
and the OOB
>>>>>>>>>>> visitor like
>>>>>>>>>>> JPA2 one takes care of dealing with these linked
beans.
>>>>>>>>>>>
>>>>>>>>>>> In the SearchBean case it is up to the custom
visitor whether to
>>>>>>>>>>> react
>>>>>>>>>>> to the '.'s or not where a '.' indicates that
for ex 'a' needs to
>>>>>>>>>>> have 'b'
>>>>>>>>>>> with a property 'c'.
>>>>>>>>>>>
>>>>>>>>>>> Do you reckon Syncope custom visitors can be
updated to support
>>>>>>>>>>> such
>>>>>>>>>>> queries ? Not sure about ElasticSearch but for
SQL it should
>>>>>>>>>>> probably be
>>>>>>>>>>> possible...
>>>>>>>>>>>
>>>>>>>>>>> Maybe in principle yes, it could be possible
to support such
>>>>>>>>>>>
>>>>>>>>>> queries but:
>>>>>>>>>>
>>>>>>>>>> 1. implementation would be rather complex as the
query logic is
>>>>>>>>>> already
>>>>>>>>>> quite involved
>>>>>>>>>> 2. we haven't had may requests for such complex queries
so far
>>>>>>>>>>
>>>>>>>>>> ...anyway, as you know, volunteers are welcome :-)
>>>>>>>>>>
>>>>>>>>>> Regards.
>>>>>>>>>>
>>>>>>>>>> On 28/06/17 09:46, Francesco Chicchiriccò wrote:
>>>>>>>>>>
>>>>>>>>>> On 28/06/2017 10:41, Sergey Beryozkin wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi
>>>>>>>>>>>>
>>>>>>>>>>>>> I think something similar works for a
CXF FIQL JPA2 visitor,
>>>>>>>>>>>>> for
>>>>>>>>>>>>> example:
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>>>>>>>>>> earch/src/test/java/org/apache/cxf/jaxrs/ext/search/jpa/JPAT
>>>>>>>>>>>>> ypedQueryVisitorFiqlTest.java#L65
>>>>>>>>>>>>>
>>>>>>>>>>>>> (find the books which have been revied
done by Ted)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Sergey,
>>>>>>>>>>>>>
>>>>>>>>>>>> that would work if we had straight beans
as in the linked
>>>>>>>>>>>> sample.
>>>>>>>>>>>>
>>>>>>>>>>>> Syncope data model is instead much more involved
as new schema
>>>>>>>>>>>> for
>>>>>>>>>>>> attributes can be defined at runtime: this
is the reason why we
>>>>>>>>>>>> have
>>>>>>>>>>>> SearchCondVisitor [1] translating FIQL into
our internal search
>>>>>>>>>>>> conditions,
>>>>>>>>>>>> which serve as input to one of available
implementations of
>>>>>>>>>>>> AnySearchDAO
>>>>>>>>>>>> like as the default one based on SQL views
[2] and another
>>>>>>>>>>>> relying on
>>>>>>>>>>>> Elasticsearch-based [3].
>>>>>>>>>>>>
>>>>>>>>>>>> Regards.
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://github.com/apache/sync
>>>>>>>>>>>> ope/blob/2_0_X/core/persistenc
>>>>>>>>>>>> e-api/src/main/java/org/apache/syncope/core/persistenc
>>>>>>>>>>>> e/api/search/SearchCondVisitor.java
>>>>>>>>>>>> [2] https://github.com/apache/sync
>>>>>>>>>>>> ope/blob/2_0_X/core/persistenc
>>>>>>>>>>>> e-jpa/src/main/java/org/apache/syncope/core/persistenc
>>>>>>>>>>>> e/jpa/dao/JPAAnySearchDAO.java
>>>>>>>>>>>> [3] https://github.com/apache/sync
>>>>>>>>>>>> ope/blob/2_0_X/ext/elasticsear
>>>>>>>>>>>> ch/persistence-jpa/src/main/java/org/apache/syncope/core/
>>>>>>>>>>>> persistence/jpa/dao/ElasticsearchAnySearchDAO.java
>>>>>>>>>>>>
>>>>>>>>>>>> On 28/06/17 08:54, Francesco Chicchiriccò
wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> On 27/06/2017 18:18, Colm O hEigeartaigh
wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks Francesco! On a related note,
let's say I have some
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> AnyObjects
>>>>>>>>>>>>>>> (Printer) with a relationship
to other AnyObjects
>>>>>>>>>>>>>>> (Cartridge).
>>>>>>>>>>>>>>> Now I
>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>> to search for a Printer which
has a relationship with a
>>>>>>>>>>>>>>> Cartridge
>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>> "colour" attribute of "blue".
Is there a way to do this via a
>>>>>>>>>>>>>>> FIQL
>>>>>>>>>>>>>>> expression?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> No, you cannot express such condition
ATM; you could do for
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> example:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> SyncopeClient.getAnyObjectSearchConditionBuilder("PRINTER").
>>>>>>>>>>>>>> inRelationships("ce75249b-76e4-44b6-88ae-0841846faceb").
>>>>>>>>>>>>>> and().is("colour").equalTo("blue").query();
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> which translates to FIQL
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> $type==PRINTER;$relationships==ce75249b-76e4-44b6-88ae-08418
>>>>>>>>>>>>>> 46faceb;colour==blue
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> but this would rather search for
blue printers having a
>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>> with an any object with key 'ce75249b-76e4-44b6-88ae-08418
>>>>>>>>>>>>>> 46faceb'.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> or alternatively
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> SyncopeClient.getAnyObjectSearchConditionBuilder("PRINTER").
>>>>>>>>>>>>>> inRelationshipTypes("WITH_CARTDRIGE").
>>>>>>>>>>>>>> and().is("color").equalTo("blue").query();
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> which translates to FIQL
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> $type==PRINTER;$relationshipTypes==WITH_CARTDRIGE;color==blue
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> but this would rather search for
blue printers having a
>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>> on type  WITH_CARTDRIGE.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jun 27, 2017 at 4:29 PM,
Francesco Chicchiriccò <
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ilgrosso@apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 27/06/2017 17:24, Colm O hEigeartaigh
wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> How can I retrieve a list
of AnyObjects? The following
>>>>>>>>>>>>>>>>> returns a
>>>>>>>>>>>>>>>>> 400:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> curl -I -X GET -u admin:password
>>>>>>>>>>>>>>>>> http://localhost:9080/syncope/rest/anyObjects
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You must at least provide
the AnyType, e.g.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> http://localhost:9080/syncope/
>>>>>>>>>>>>>>>> rest/anyObjects;fiql=%24type%3
>>>>>>>>>>>>>>>> D%3DPRINTER
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
> --
> Francesco Chicchiriccò
>
> Tirasa - Open Source Excellence
> http://www.tirasa.net/
>
> Member at The Apache Software Foundation
> Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
> http://home.apache.org/~ilgrosso/
>
>


-- 
Colm O hEigeartaigh

Talend Community Coder
http://coders.talend.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message