Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 7E587200B50 for ; Fri, 29 Jul 2016 16:47:01 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 7CF21160A79; Fri, 29 Jul 2016 14:47:01 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 9A9CF160A6E for ; Fri, 29 Jul 2016 16:47:00 +0200 (CEST) Received: (qmail 74772 invoked by uid 500); 29 Jul 2016 14:46:57 -0000 Mailing-List: contact dev-help@syncope.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@syncope.apache.org Delivered-To: mailing list dev@syncope.apache.org Received: (qmail 74643 invoked by uid 99); 29 Jul 2016 14:46:57 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 29 Jul 2016 14:46:57 +0000 Received: from [192.168.0.7] (unknown [217.133.18.16]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 031B51A003E for ; Fri, 29 Jul 2016 14:46:56 +0000 (UTC) Subject: Re: [DISCUSS] Improving Camel-based provisioning To: dev@syncope.apache.org References: <9e7aeae9-7438-1634-520c-7cac27fc4d7e@apache.org> <1468845557.6079.15.camel@tirasa.net>

<7ef7dc73-81a8-147c-ce9d-70690fb43085@apache.org> <6e197b60-2ac1-faf2-d70d-8217e99937a4@apache.org> <804d1de3-1344-cbba-99cd-33058a99c0ac@apache.org>

<64341be6-050f-b497-c949-3a16e0aa1d23@apache.org> From: =?UTF-8?Q?Francesco_Chicchiricc=c3=b2?= Message-ID: <3d0ae5a0-c6fb-9b0a-ec2d-030f0a730e9c@apache.org> Date: Fri, 29 Jul 2016 16:46:52 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable archived-at: Fri, 29 Jul 2016 14:47:01 -0000 On 29/07/2016 16:30, Colm O hEigeartaigh wrote: > Ok I've committed the initial change to use the new Camel component and= > would welcome some feedback. Instead of referencing random Camel Proces= sors > in the routes, we now have a single component called "propagate". All o= f > the routes call something like: > > ?anyType=3D&options"/> This consolidation overall is extremely interesting, thanks for taking=20 this forward. > PropagateType is one of the following: create, update, delete, provisio= n, > deprovision, status, suspend, confirmPasswordReset > AnyType is one of: user, group, any As already noticed via IRC, org.apache.syncope.core.provisioning.camel.AnyType looks pretty much the same as org.apache.syncope.common.lib.types.AnyTypeKind so I'd suggest to remove the former and to use the latter instead,=20 possibly changing ?anyType=3D&options"/> to ?anyTypeKind=3D&options"= /> e.g. the parameter name as well, since "anyType" has a different meaning = (can be PRINTER, SERVICE, whatever you might want to define on your=20 Syncope deployment). > The only "option" that's currently supported is "pull" which if true > supports some of the functionality that was available in the processors= , > e.g. user update + pull. So for example: > > > > Thoughts? Criticisms? One thing I was a bit unsure of is the "pull" > functionality in the StatusProducer. It doesn't follow the other patter= ns > in terms of the PropagationManager + TaskExecutor, but instead uses the= > UserWorkflowAdapter. Should this functionality be put somewhere else I > wonder? Not sure, but the Java Provisioning Manager counterpart is https://github.com/apache/syncope/blob/master/core/provisioning-java/src/= main/java/org/apache/syncope/core/provisioning/java/DefaultUserProvisioni= ngManager.java#L159-L178 e.g. a branch in the user update, which anyway proceeds with propagation = in the lines afterwards. Regards. > On Wed, Jul 27, 2016 at 8:21 AM, Francesco Chicchiricc=C3=B2 wrote: > >> Hi Colm, >> >> I guess the routes look a bit nicer as well as we're calling the same >>> component rather than individual processors etc. >>> >> Definitely, +1 >> >> Actually I think I can have just a single implementation per >>> user/group/etc, just by checking to see what the type of the Object s= tored >>> on the exchange is, so something like the following would work for >>> user/group/any etc.: >>> >>> >>> >> That's nice improvement as well. >> >> Thanks: are you going to open an improvement on JIRA for your work? >> Regards. >> >> >> On Tue, Jul 26, 2016 at 5:35 PM, Colm O hEigeartaigh >>> wrote: >>> >>> Thanks Francesco! >>>> I did a quick POC for the user create route + got it working locally= =2E Any >>>> thoughts on what the route should look like? I could create a separa= te >>>> component for each of the user/groups/any etc., so the route would l= ook >>>> something like: >>>> >>>> >>>> >>>> >>>> Or I could have a single component that does something like: >>>> >>>> >>>> >>>> >>>> etc. WDYT? >>>> >>>> BTW I'm not sure that this change is really buying us much improveme= nt, >>>> as >>>> the java logic looks more or less the same, from what I've done so f= ar. I >>>> guess one improvement is that we do away with all of the @Component = Camel >>>> Processor implementations (instead replacing them with Camel >>>> DefaultProducer implementations that are controlled by the component= ). I >>>> guess the routes look a bit nicer as well as we're calling the same >>>> component rather than individual processors etc. >>>> >>>> Colm. >>>> >>>> On Tue, Jul 26, 2016 at 11:54 AM, Francesco Chicchiricc=C3=B2 < >>>> ilgrosso@apache.org> wrote: >>>> >>>> Hi, >>>>> FYI I have just committed >>>>> >>>>> >>>>> >>>>> https://git1-us-west.apache.org/repos/asf?p=3Dsyncope.git;a=3Dcommi= t;h=3D945be877 >>>>> >>>>> a modification that should be simplifying the usage >>>>> PropagationTaskExecutor / PropagationReporter >>>>> >>>>> Regards. >>>>> >>>>> >>>>> On 22/07/2016 13:45, Colm O hEigeartaigh wrote: >>>>> >>>>> Hi Francesco, >>>>>> I think a dedicated feature branch will not be necessary. I'll pro= bably >>>>>> do >>>>>> it over a few commits, maybe do an operation at a time so as not t= o >>>>>> break >>>>>> the tests. >>>>>> >>>>>> Colm. >>>>>> >>>>>> On Fri, Jul 22, 2016 at 7:59 AM, Francesco Chicchiricc=C3=B2 < >>>>>> ilgrosso@apache.org>wrote: >>>>>> >>>>>> Hi Colm, >>>>>> >>>>>>> as it seems that Giacomo as well is happy if you can take this ta= sk, >>>>>>> can >>>>>>> you please describe how are you going to work on it? Dedicated fe= ature >>>>>>> branch? Or you expect to be simple enough to stay in a single com= mit? >>>>>>> >>>>>>> Thanks! >>>>>>> Regards. >>>>>>> >>>>>>> >>>>>>> On 20/07/2016 13:45, Francesco Chicchiricc=C3=B2 wrote: >>>>>>> >>>>>>> On 20/07/2016 13:17, Colm O hEigeartaigh wrote: >>>>>>> >>>>>>>> Hi Francesco, >>>>>>>> >>>>>>>>> It should be fairly straightforward I'd say. Is there reasonabl= e >>>>>>>>> test >>>>>>>>> coverage of the camel routes in the build? >>>>>>>>> >>>>>>>>> As you can see from [3] the "all" profile (featuring Activiti, = Camel >>>>>>>>> >>>>>>>> and >>>>>>>> Swagger) is active by default, so the whole integration test sui= te is >>>>>>>> run >>>>>>>> with Camel routes by default. >>>>>>>> >>>>>>>> I'd like to volunteer to take it on, given that I plan on talkin= g >>>>>>>> about >>>>>>>> >>>>>>>> Syncope + Camel, unless you or Giacomo would like to implement i= t? >>>>>>>>> This sounds great to me, and I would also say that there is no >>>>>>>>> >>>>>>>> particular >>>>>>>> rush to finish before releasing 2.0.0: such an improvement can a= lso >>>>>>>> come >>>>>>>> afterwards (2.0.1, 2.0.2, ...). >>>>>>>> >>>>>>>> Regards. >>>>>>>> >>>>>>>> On Wed, Jul 20, 2016 at 12:03 PM, Francesco Chicchiricc=C3=B2 < >>>>>>>> >>>>>>>> ilgrosso@apache.org> wrote: >>>>>>>>> On 19/07/2016 17:46, Colm O hEigeartaigh wrote: >>>>>>>>> >>>>>>>>> Hi Francesco, >>>>>>>>>> How do you envisage this change would be made? The Processors = in >>>>>>>>>>> question >>>>>>>>>>> pretty much all call the PropagationManager to create some ta= sks >>>>>>>>>>> and >>>>>>>>>>> then >>>>>>>>>>> execute them using the PropagationTaskExecutor. We could crea= te a >>>>>>>>>>> new >>>>>>>>>>> Camel >>>>>>>>>>> component to encapsulate all of this functionality, and then = just >>>>>>>>>>> refer to >>>>>>>>>>> it in the Spring DSL. Something like ">>>>>>>>>> uri=3D"propagate:create?...>", >>>>>>>>>>> ">>>>>>>>>> uri=3D"propagate:update?...>" etc. Are you thinking alone the= se >>>>>>>>>>> lines >>>>>>>>>>> or >>>>>>>>>>> something else? >>>>>>>>>>> >>>>>>>>>>> Hi Colm, >>>>>>>>>>> >>>>>>>>>>> considering my (very limited) understanding of Camel, I would= have >>>>>>>>>> expected exactly something like this. >>>>>>>>>> >>>>>>>>>> How difficult would it be to implement? >>>>>>>>>> >>>>>>>>>> Regards. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Jul 18, 2016 at 1:39 PM, Giacomo Lamonaco < >>>>>>>>>> >>>>>>>>>> giacomo.lamonaco@tirasa.net> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Francesco, >>>>>>>>>>> >>>>>>>>>>> I think it would be great! Currently camel routes are defined= >>>>>>>>>>> using >>>>>>>>>>> >>>>>>>>>>>> Spring DSL: as you can image we need to understand if = the >>>>>>>>>>>> logic you >>>>>>>>>>>> described can be expressed using that DSL. IMHO that's not a= >>>>>>>>>>>> difficult >>>>>>>>>>>> task and it would be great to develop a POC. Otherwise we ca= n >>>>>>>>>>>> investigate other DSLs (equivalent to Spring DSL of course).= >>>>>>>>>>>> >>>>>>>>>>>> Best Regards, >>>>>>>>>>>> Giacomo >>>>>>>>>>>> >>>>>>>>>>>> Il giorno ven, 15/07/2016 alle 18.16 +0200, Francesco >>>>>>>>>>>> Chicchiricc=C3=B2 >>>>>>>>>>>> ha >>>>>>>>>>>> scritto: >>>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> as you know, Camel-based provisioning is one of the coolest >>>>>>>>>>>>> features >>>>>>>>>>>>> among the several cool features in 2.0. >>>>>>>>>>>>> >>>>>>>>>>>>> The implementation is essentially done this way: each metho= d in >>>>>>>>>>>>> CamelUserProvisioningManager [1] (and similarly for groups = and >>>>>>>>>>>>> any >>>>>>>>>>>>> objects) invokes some Camel route, then at the end of the r= oute, >>>>>>>>>>>>> some >>>>>>>>>>>>> 'processor' is invoked (as [2] for example, when user is >>>>>>>>>>>>> created). >>>>>>>>>>>>> >>>>>>>>>>>>> As you might see from [2] and all similar classes in that >>>>>>>>>>>>> package, >>>>>>>>>>>>> there >>>>>>>>>>>>> is still some relevant logic implemented in Java, which mig= ht be >>>>>>>>>>>>> moved >>>>>>>>>>>>> back to the route definition, hence increasing the general >>>>>>>>>>>>> benefits >>>>>>>>>>>>> of >>>>>>>>>>>>> the whole concept of Camel-based provisioning. >>>>>>>>>>>>> >>>>>>>>>>>>> Do you also see this as an enhancement? Do you think it is >>>>>>>>>>>>> possible >>>>>>>>>>>>> / >>>>>>>>>>>>> feasible to implement with limited effort? >>>>>>>>>>>>> >>>>>>>>>>>>> Regards. >>>>>>>>>>>>> >>>>>>>>>>>>> [1] >>>>>>>>>>>>> https://github.com/apache/syncope/blob/master/ext/camel/pro= visioning-camel/src/main/java/org/apache/syncope/core/provisioning/camel/= CamelUserProvisioningManager.java >>>>>>>>>>>>> [2] >>>>>>>>>>>>> https://github.com/apache/syncope/blob/master/ext/camel/pro= visioning-camel/src/main/java/org/apache/syncope/core/provisioning/camel/= processor/UserCreateProcessor.java >>>>>>>>>>>>> [3] >>>>>>>>>>>>> https://github.com/apache/syncope/blob/master/fit/core-refe= rence/pom.xml#L963 --=20 Francesco Chicchiricc=C3=B2 Tirasa - Open Source Excellence http://www.tirasa.net/ Involved at The Apache Software Foundation: member, Syncope PMC chair, Cocoon PMC, Olingo PMC, CXF Committer, OpenJPA Committer, PonyMail PPMC http://home.apache.org/~ilgrosso/