oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Ratnakar <varunratna...@gmail.com>
Subject Re: [DISCUSS} Making Wings a part of OODT
Date Thu, 13 Nov 2014 09:15:47 GMT
Thanks for looking into this Rishi.

> Hey Chris & Varun, how much of: /core, /ontapi, /planner, /opmm, etc. represent customized
code and are effectively different than the Wings central codebase on GitHub [1]? If its exactly
the same code,

Currently, it is exactly the same code. We enable OODT via a configuration option (and throwing
in the oodt-adaptor jar into the wings-portal webapp)


> would using Maven Overlays reduce the repetition of code? Also, perhaps the Wings project
may want to consider publishing artifacts to Maven central so that JARs can be pulled down.
All in all, I’m just wondering if we really should be placing the entire Wings codebase
into OODT versus just extensions specifically for OODT and “pulling down” the rest. 
> 
> Thoughts?
> 
> Thanks,
> Rishi
> 
> [1] https://github.com/IKCAP/wings
> [2] http://maven.apache.org/plugins/maven-war-plugin/overlays.html
> 
> On Nov 12, 2014, at 7:57 AM, Mattmann, Chris A (3980) <chris.a.mattmann@jpl.nasa.gov>
wrote:
> 
>> Thanks Varun,
>> 
>> 
>> Thanks for making an RB. In looking at some of
>> this, I realized that we probably need to think about
>> organization of how Wings will become a part. It looks
>> like Wings includes both web app and core components.
>> 
>> 
>> 1. For the wings backend, I would suggest creating a
>> http://svn.apache.org/repos/asf/oodt/trunk/wings
>> folder and then making:
>> /core
>> /ontapi
>> /planner
>> /opmm
>> 
>> folders underneath.
>> 
>> 2. For the wings portal I would suggest creating
>> a http://svn.apache.org/repos/asf/oodt/trunk/webapp/wings
>> directory for the wings webapp-portal.
>> 
>> Does this make sense? Also you are going to see a bunch of
>> comments from me regarding updates to groupIds and artifactIds.
>> Group ids should follow the org.apache.oodt namespace and have
>> that as a prefix.
>> 
>> More info to come as I review more :)
>> 
>> Others, feel free to chime in.
>> 
>> Cheers,
>> Chris
>> 
>> -----Original Message-----
>> From: Varun Ratnakar <varunratnakar@gmail.com>
>> Reply-To: <dev@oodt.apache.org>
>> Date: Wednesday, November 12, 2014 at 4:47 PM
>> To: <dev@oodt.apache.org>
>> Cc: "Verma, Rishi (398M)" <Rishi.Verma@jpl.nasa.gov>
>> Subject: Re: Making Wings a part of OODT
>> 
>>> Jira: https://issues.apache.org/jira/browse/OODT-795
>>> <https://issues.apache.org/jira/browse/OODT-795>Review Board:
>>> https://reviews.apache.org/r/27915/ <https://reviews.apache.org/r/27915/>
>>> 
>>> 
>>> 
>>>> On 12 Nov 2014, at 15:26, Mattmann, Chris A (3980)
>>>> <chris.a.mattmann@jpl.nasa.gov> wrote:
>>>> 
>>>> hahah bring it on! :) will really enjoy reviewing it thanks
>>>> Varun
>>>> 
>>>> 
>>>> 
>>>> -----Original Message-----
>>>> From: Varun Ratnakar <varunratnakar@gmail.com
>>>> <mailto:varunratnakar@gmail.com>>
>>>> Reply-To: <dev@oodt.apache.org <mailto:dev@oodt.apache.org>>
>>>> Date: Wednesday, November 12, 2014 at 4:24 PM
>>>> To: <dev@oodt.apache.org <mailto:dev@oodt.apache.org>>
>>>> Cc: "Verma, Rishi (398M)" <Rishi.Verma@jpl.nasa.gov
>>>> <mailto:Rishi.Verma@jpl.nasa.gov>>
>>>> Subject: Re: Making Wings a part of OODT
>>>> 
>>>>> Thanks Chris ! We are very excited to make Wings available to the OODT
>>>>> audience as well !
>>>>> I will work on creating a JIRA issue and uploading the patch on RB.
>>>>> 
>>>>> Warning: It is going to be one BIG patch :-) (~100K Lines)
>>>>> 
>>>>> Regards,
>>>>> Varun
>>>>> 
>>>>> 
>>>>>> On 12 Nov 2014, at 11:11, Mattmann, Chris A (3980)
>>>>>> <chris.a.mattmann@jpl.nasa.gov> wrote:
>>>>>> 
>>>>>> Varun this sounds like a fantastic idea and we would welcome Wings
>>>>>> as a part of Apache OODT. I think your strategy of making wings a
>>>>>> webapp/wings module and integrating some of the other code elements
>>>>>> in the ways you¹ve proposed are great. You¹re already an Apache
OODT
>>>>>> PMC member and committer and you know the drill. I think a move this
>>>>>> big should result in some review boards showing the integration.
I
>>>>>> would be happy to be a review person for the patch on RB and in moving
>>>>>> this forward. Maybe we should create a JIRA issue for this and look
>>>>>> forward
>>>>>> to the review!
>>>>>> 
>>>>>> Cheers,
>>>>>> Chris
>>>>>> 
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> Chris Mattmann, Ph.D.
>>>>>> Chief Architect
>>>>>> Instrument Software and Science Data Systems Section (398)
>>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>>>> Office: 168-519, Mailstop: 168-527
>>>>>> Email: chris.a.mattmann@nasa.gov <mailto:chris.a.mattmann@nasa.gov>
>>>>>> <mailto:chris.a.mattmann@nasa.gov <mailto:chris.a.mattmann@nasa.gov>>
>>>>>> WWW:  http://sunset.usc.edu/~mattmann/
>>>>>> <http://sunset.usc.edu/~mattmann/>
>>>>>> <http://sunset.usc.edu/~mattmann/ <http://sunset.usc.edu/~mattmann/>>
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> Adjunct Associate Professor, Computer Science Department
>>>>>> University of Southern California, Los Angeles, CA 90089 USA
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Varun Ratnakar <varunr@isi.edu <mailto:varunr@isi.edu>
>>>>>> <mailto:varunr@isi.edu <mailto:varunr@isi.edu>>>
>>>>>> Reply-To: "dev@oodt.apache.org <mailto:dev@oodt.apache.org>
>>>>>> <mailto:dev@oodt.apache.org <mailto:dev@oodt.apache.org>>"
>>>>>> <dev@oodt.apache.org <mailto:dev@oodt.apache.org>
>>>>>> <mailto:dev@oodt.apache.org <mailto:dev@oodt.apache.org>>>
>>>>>> Date: Wednesday, November 12, 2014 at 10:06 AM
>>>>>> To: "Verma, Rishi (398M)" <Rishi.Verma@jpl.nasa.gov
>>>>>> <mailto:Rishi.Verma@jpl.nasa.gov>
>>>>>> <mailto:Rishi.Verma@jpl.nasa.gov <mailto:Rishi.Verma@jpl.nasa.gov>>>
>>>>>> Cc: "dev@oodt.apache.org <mailto:dev@oodt.apache.org>
>>>>>> <mailto:dev@oodt.apache.org <mailto:dev@oodt.apache.org>>"
>>>>>> <dev@oodt.apache.org <mailto:dev@oodt.apache.org>
>>>>>> <mailto:dev@oodt.apache.org <mailto:dev@oodt.apache.org>>>
>>>>>> Subject: Re: Making Wings a part of OODT
>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Hi Rishi.
>>>>>>> 
>>>>>>> 
>>>>>>> An update: I just committed 2 patches to the OODT svn repository
in
>>>>>>> order
>>>>>>> to allow the Wings-OODT integration.
>>>>>>> 1. https://reviews.apache.org/r/14626/
>>>>>>> <https://reviews.apache.org/r/14626/>
>>>>>>> 2. https://reviews.apache.org/r/15936/
>>>>>>> <https://reviews.apache.org/r/15936/>
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> As of now, Wings (http://www.wings-workflows.org/
>>>>>>> <http://www.wings-workflows.org/>) is able to integrate
>>>>>>> with OODT trunk via the following interfaces:
>>>>>>> 
>>>>>>> 
>>>>>>> - Interfaces with File Manager via fmclient & extensions
to
>>>>>>> cas-curator
>>>>>>> (patch 1 above)
>>>>>>> 
>>>>>>> 
>>>>>>> - Interfaces with Workflow Manager via wmclient and some extensions
>>>>>>> (patch 2 above) and WMServices*
>>>>>>> - *WMServices is a webapp that is currently part of the Wings
OODT
>>>>>>> engine
>>>>>>> extensions, but we should ideally put it into [oodt]/webapp directory
>>>>>>> later
>>>>>>> - Currently code is at
>>>>>>> https://github.com/IKCAP/wings/tree/master/engines/oodt/wmservices
>>>>>>> <https://github.com/IKCAP/wings/tree/master/engines/oodt/wmservices>
>>>>>>> - *WMServices interfaces with the Workflow Manager via the
>>>>>>> ³PackagedWorkflowRepository² and uses it to add tasks/workflows
on
>>>>>>> the
>>>>>>> fly as requested by Wings
>>>>>>> 
>>>>>>> 
>>>>>>> - Interfaces with OODT Workflow Engine via wings-oodt-task
>>>>>>> - wings-oodt-task is an implementation of a WorkflowTaskInstance,
and
>>>>>>> uses some extra configuration properties to run Wings components.
>>>>>>> - wings-oodt-task-<version>.jar needs to be put in
>>>>>>> [WORKFLOW_HOME]/lib
>>>>>>> - wings-oodt-task requires the file manager as well, so
>>>>>>> cas-filemgr-<version>.jar also needs to be put in [WORKFLOW_HOME]/lib
>>>>>>> - Code is at 
>>>>>>> https://github.com/IKCAP/wings/tree/master/engines/oodt/task
>>>>>>> <https://github.com/IKCAP/wings/tree/master/engines/oodt/task>
>>>>>>> <https://github.com/IKCAP/wings/tree/master/engines/oodt/task
>>>>>>> <https://github.com/IKCAP/wings/tree/master/engines/oodt/task>>
>>>>>>> 
>>>>>>> 
>>>>>>> (Copying over the extension diagram again for the benefit of
others
>>>>>>> on
>>>>>>> the mailing list. ‹ Note this doesn¹t show the file manager
>>>>>>> integration)
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> We would like to move forward now, and make Wings a part of the
OODT
>>>>>>> source tree. Wings is currently on Github and is already a maven
>>>>>>> project,
>>>>>>> so I imagine that the move shouldn¹t be too hard. The code is
>>>>>>> currently
>>>>>>> at https://github.com/IKCAP/wings <https://github.com/IKCAP/wings>
>>>>>>> <https://github.com/IKCAP/wings <https://github.com/IKCAP/wings>>
>>>>>>> 
>>>>>>> 
>>>>>>> If I just commit the wings code to [oodt]/webapp/wings directory
(and
>>>>>>> add
>>>>>>> wings as a module in the main pom.xml), would that be sufficient
? I
>>>>>>> would look into integration with RADiX after the code is committed.
>>>>>>> 
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Varun
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 7 Nov 2014, at 22:22, Varun Ratnakar <varunr@isi.edu
>>>>>>> <mailto:varunr@isi.edu>
>>>>>>> <mailto:varunr@isi.edu <mailto:varunr@isi.edu>>>
wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> This is great! OODT could really benefit by having the Wings
>>>>>>> framework
>>>>>>> integrated (especially to create/modify workflows graphically).
Could
>>>>>>> you
>>>>>>> tell me where you are at and the extension points you¹re looking
at?
>>>>>>> I¹m
>>>>>>> sure you¹ve gone through
>>>>>>> this many times already, but just in case, the ³extension points²
>>>>>>> section of the below link is a good place to start with any
>>>>>>> integration
>>>>>>> work.
>>>>>>> 
>>>>>>> 
>>>>>>> http://oodt.apache.org/components/maven/workflow/development/developer
>>>>>>> .h 
>>>>>>> <http://oodt.apache.org/components/maven/workflow/development/develope
>>>>>>> r.h>
>>>>>>> tm 
>>>>>>> 
>>>>>>> <http://oodt.apache.org/components/maven/workflow/development/develope
>>>>>>> r 
>>>>>>> <http://oodt.apache.org/components/maven/workflow/development/develope
>>>>>>> r>.
>>>>>>> htm>
>>>>>>> l
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> I¹ve modified the diagram in the link above to explain where
I'm at.
>>>>>>> Explanation & Details below the diagram.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> <snip..>
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> The Wings OODT Engine consists of an Adapter (Jar file imported
by
>>>>>>> the
>>>>>>> Wings Portal), a Wings OODT Task ( jar that needs to be installed
>>>>>>> into
>>>>>>> [WORKFLOW_HOME]/lib ), and WMServices ( a webapp that needs to
be run
>>>>>>> in
>>>>>>> the same server as the Workflow
>>>>>>> Manager ).
>>>>>>> 
>>>>>>> 
>>>>>>> Note 1: The Adapter also connects to CAS-Curator & FMClient,
but I¹ve
>>>>>>> omitted them from the diagram as they don¹t pertain to the Workflow
>>>>>>> Manager. 
>>>>>>> Note 2: Wings OODT Task needs cas-filemgr-<version>.jar
to be present
>>>>>>> in
>>>>>>> [WORKFLOW_HOME]/lib
>>>>>>> 
>>>>>>> 
>>>>>>> The wings-oodt engine code can be found here:
>>>>>>> https://github.com/IKCAP/wings/tree/master/engines/oodt
>>>>>>> <https://github.com/IKCAP/wings/tree/master/engines/oodt>
>>>>>>> <https://github.com/IKCAP/wings/tree/master/engines/oodt
>>>>>>> <https://github.com/IKCAP/wings/tree/master/engines/oodt>>
( Wings
>>>>>>> github
>>>>>>> root: https://github.com/IKCAP/wings
>>>>>>> <https://github.com/IKCAP/wings> <https://github.com/IKCAP/wings
>>>>>>> <https://github.com/IKCAP/wings>> )
>>>>>>> 
>>>>>>> 
>>>>>>> For this to work, there are 2 patches to OODT required that I
haven¹t
>>>>>>> yet
>>>>>>> committed.
>>>>>>> https://reviews.apache.org/r/14626/
>>>>>>> <https://reviews.apache.org/r/14626/>
>>>>>>> <https://reviews.apache.org/r/14626/
>>>>>>> <https://reviews.apache.org/r/14626/>>
>>>>>>> https://reviews.apache.org/r/15936/
>>>>>>> <https://reviews.apache.org/r/15936/>
>>>>>>> <https://reviews.apache.org/r/15936/
>>>>>>> <https://reviews.apache.org/r/15936/>>
>>>>>>> 
>>>>>>> 
>>>>>>> I had put up another patch for the addition of a wmservices webapp
>>>>>>> (https://reviews.apache.org/r/15937/
>>>>>>> <https://reviews.apache.org/r/15937/>
>>>>>>> <https://reviews.apache.org/r/15937/
>>>>>>> <https://reviews.apache.org/r/15937/>>), but for now,
I have moved
>>>>>>> this
>>>>>>> into the Wings-OODT Engine code.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> I¹ve been working a lot on OODT RADiX and promoting that as
an easier
>>>>>>> way
>>>>>>> to ³use² OODT out-of-the-box. I¹d be really interested in
plugging
>>>>>>> Wings
>>>>>>> as a component of RADiX so that users can use Wings & OODT
without
>>>>>>> much
>>>>>>> configuration themselves.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Sounds great Rishi !
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> When integrating the Solr search engine in OODT RADiX as the
default
>>>>>>> File
>>>>>>> Manager catalog (something I did a few months back), here¹s
what we
>>>>>>> did:
>>>>>>> Manually integrating wiki (this was how someone would do it
>>>>>>> themselves):
>>>>>>> 
>>>>>>> 
>>>>>>> https://cwiki.apache.org/confluence/display/OODT/Integrating+Solr+with
>>>>>>> +O 
>>>>>>> <https://cwiki.apache.org/confluence/display/OODT/Integrating+Solr+wit
>>>>>>> h+O>
>>>>>>> OD 
>>>>>>> 
>>>>>>> <https://cwiki.apache.org/confluence/display/OODT/Integrating+Solr+wit
>>>>>>> h+ 
>>>>>>> <https://cwiki.apache.org/confluence/display/OODT/Integrating+Solr+wit
>>>>>>> h+>
>>>>>>> OOD>
>>>>>>> T+RADiX
>>>>>>> Automatically (we built Solr as an optional catalog to use):
>>>>>>> https://issues.apache.org/jira/browse/OODT-707
>>>>>>> <https://issues.apache.org/jira/browse/OODT-707>
>>>>>>> <https://issues.apache.org/jira/browse/OODT-707
>>>>>>> <https://issues.apache.org/jira/browse/OODT-707>>
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> I shall take a look.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Once things are ready from the Wings/OODT integration side, let
me
>>>>>>> know,
>>>>>>> and it would be great to merge it into RADiX so people can use
it
>>>>>>> right
>>>>>>> away and set it as an option for people to plug their Workflow
>>>>>>> Manager
>>>>>>> into. 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> I think we are almost there regarding the Wings/OODT integration
from
>>>>>>> my
>>>>>>> side. Just need to commit those 2 patches and put the wings code
into
>>>>>>> the
>>>>>>> OODT codebase somehow.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Do let me know if I can bother you in the next couple of weeks,
or if
>>>>>>> there is someone else I could talk to about this.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Yes you can bother me anytime - although, as a recommendation,
I¹d
>>>>>>> suggest CCing
>>>>>>> dev@oodt.apache.org <mailto:dev@oodt.apache.org>
>>>>>>> <mailto:dev@oodt.apache.org <mailto:dev@oodt.apache.org>>
in all
>>>>>>> e-mails so
>>>>>>> that others can weigh in and
>>>>>>> provide help as well!
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> I would take you up on that :-) .. And yes I will CC
>>>>>>> dev@oodt.apache.org <mailto:dev@oodt.apache.org>
>>>>>>> <mailto:dev@oodt.apache.org <mailto:dev@oodt.apache.org>>
>>>>>>> as you recommended.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Warm Regards,
>>>>>>> Varun
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Thanks Varun,
>>>>>>> All the best.
>>>>>>> 
>>>>>>> 
>>>>>>> Rishi
>>>>>>> 
>>>>>>> On Nov 5, 2014, at 3:30 AM, Varun Ratnakar <varunr@isi.edu
>>>>>>> <mailto:varunr@isi.edu>
>>>>>>> <mailto:varunr@isi.edu <mailto:varunr@isi.edu>>>
wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> Hi Rishi,
>>>>>>> 
>>>>>>> 
>>>>>>> How¹ve you been ?! Wanted to catch up with you and find out
if you
>>>>>>> would
>>>>>>> have time in the next few weeks to help out (mostly answering
>>>>>>> questions)
>>>>>>> with making Wings a part of OODT. Chris had suggested that we
should
>>>>>>> start by just making Wings as
>>>>>>> part of OODT codebase a while ago. We have been working on adding
>>>>>>> lots
>>>>>>> of new features into Wings, and now that things are settling
down, we
>>>>>>> would really like to push this. I wanted to get an overview of
what
>>>>>>> all
>>>>>>> is involved in making that happen. Any pointers
>>>>>>> would be helpful !
>>>>>>> 
>>>>>>> 
>>>>>>> Do let me know if I can bother you in the next couple of weeks,
or if
>>>>>>> there is someone else I could talk to about this.
>>>>>>> 
>>>>>>> 
>>>>>>> Warm Regards,
>>>>>>> Varun
>>>>>>> 
>>>>>>> 
>>>>>>> PS: In case you¹re interested, you can find about the new wings
>>>>>>> features
>>>>>>> here: 
>>>>>>> http://wings-workflows.org/node/17664
>>>>>>> <http://wings-workflows.org/node/17664>
>>>>>>> <http://wings-workflows.org/node/17664
>>>>>>> <http://wings-workflows.org/node/17664>> &&
>>>>>>> http://wings-workflows.org/tutorial
>>>>>>> <http://wings-workflows.org/tutorial>
>>>>>>> <http://wings-workflows.org/tutorial
>>>>>>> <http://wings-workflows.org/tutorial>>
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ---
>>>>>>> Rishi Verma
>>>>>>> NASA Jet Propulsion Laboratory
>>>>>>> California Institute of Technology
>>>>>>> 4800 Oak Grove Drive, M/S 158-248
>>>>>>> Pasadena, CA 91109
>>>>>>> Tel: 1-818-393-5826
>>> 
>> 
> 
> ---
> Rishi Verma
> NASA Jet Propulsion Laboratory
> California Institute of Technology
> 


Mime
View raw message