oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (3980)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: workflow/lib and resource/lib jar files.
Date Fri, 11 Sep 2015 05:56:29 GMT
To clarify this, since I was dictating on my cell while sending this.

1. The design of the workflow manager is so that it doesn’t need
CAS-PGE. You need it Val, b/c you are using CAS-PGE and combining
workflow, resmgr, filemgr, and crawler. That’s the point of CAS-PGE.
However, if that came standard in workflow manager, it would be a
cyclic dependency nightmare.

2. Workflow doesn’t require CAS-PGE, nor file manager, nor crawler,
etc. It only requires Resource Manager (as an optional dependency
if you submit jobs to it). The 3 service components of OODT were
carefully designed to have this independence from one another (e.g.,
file manager doesn’t depend on workflow; workflow only [optionally]
depends on resource manager; resource manager doesn’t depend on FM
or WM). 

3. You can use Workflow manager to manage control flow and data flow
(send first run this script, then send an email) and it works fine.

4. RADIX should DEFINITELY have these dependencies right. It’s meant
to give you a full stack OODT deployment including CAS-PGE. If it
doesn’t have the deps right, we can fix RADIX.


Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA

-----Original Message-----
From: jpluser <chris.a.mattmann@jpl.nasa.gov>
Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org>
Date: Thursday, September 10, 2015 at 3:31 PM
To: "<dev@oodt.apache.org>" <dev@oodt.apache.org>
Subject: Re: workflow/lib and resource/lib jar files.

>I guys I would strongly advise against making hash workflow is separated
>from PGE for reason it has to do with the click dependencies and also
>because the standard work let manager out of the box still can manage
>controlling dataflow it doesn't need a file management or crawler or
>things like that
>Sent from my iPhone
>> On Sep 10, 2015, at 3:27 PM, Mallder, Valerie
>><Valerie.Mallder@jhuapl.edu> wrote:
>> Sure, I'll do it next week. I'm off tomorrow.  It will be nice to have
>>this fixed!
>> Val
>> Valerie A. Mallder
>> New Horizons Deputy Mission System Engineer
>> Johns Hopkins University/Applied Physics Laboratory
>>> -----Original Message-----
>>> From: Lewis John Mcgibbney [mailto:lewis.mcgibbney@gmail.com]
>>> Sent: Thursday, September 10, 2015 5:48 PM
>>> To: dev@oodt.apache.org
>>> Subject: Re: workflow/lib and resource/lib jar files.
>>> Hi Val,
>>> I think your right!
>>> lmcgibbn@LMC-032857
>>> $ ls
>>> antlr-2.7.6.jar                commons-collections-2.1.jar
>>> geronimo-activation_1.1_spec-1.1.jar    junit-3.8.2.jar
>>> tika-core-1.10.jar
>>> aopalliance-1.0.jar            commons-dbcp-1.2.1.jar
>>> geronimo-javamail_1.4_mail-1.8.1.jar    lucene-core-2.0.0.jar
>>> xercesImpl-2.9.1.jar
>>> asm-1.5.3.jar                commons-httpclient-3.0.jar
>>> guava-10.0.1.jar            oodt-commons-0.11-SNAPSHOT.jar
>>> xml-apis-1.0.b2.jar
>>> asm-attrs-1.5.3.jar            commons-io-1.4.jar
>>> hibernate-3.2.5.ga.jar            pcs-input-0.11-SNAPSHOT.jar
>>> xmlrpc-2.0.1.jar
>>> cas-cli-0.11-SNAPSHOT.jar        commons-lang-2.4.jar
>>> hsqldb-            spring-beans-2.0.8.jar
>>> xpp3_min-1.1.4c.jar
>>> cas-metadata-0.11-SNAPSHOT.jar        commons-logging-1.0.3.jar
>>> httpclient-4.2.1.jar            spring-context-2.0.8.jar
>>> xstream-1.3.1.jar
>>> cas-resource-0.11-SNAPSHOT.jar        commons-pool-1.2.jar
>>> httpcore-4.2.1.jar            spring-core-2.5.4.jar
>>> cas-workflow-0.11-SNAPSHOT.jar        concurrent-1.3.4.jar
>>> joda-time-2.5.jar            spring-dao-2.0.8.jar
>>> cglib-2.1_3.jar                dom4j-1.6.1.jar
>>> jsr305-1.3.9.jar            spring-hibernate3-2.0.8.jar
>>> commons-codec-1.3.jar            ehcache-1.2.3.jar
>>> jug-2.0.0-asl.jar            spring-jdbc-2.0.8.jar
>>> No PGETaskInstance (from cas-pge-0.x.jar), ProductCrawler (from
>>> cas-crawler-0.x.jar) and CoreMetKeys (from cas-filemgr-0.x.jar).
>>> The libraries should be defined within the specified pom.xml files
>>>with scope set to
>>> runtime e.g within workflow/pom.xml you should define
>>>    <dependency>
>>>      <groupId>org.apache.oodt</groupId>
>>>      <artifactId>cas-pge</artifactId>
>>>      <version>${project.parent.version}</version>
>>>      <scope>runtime</scope>
>>>    </dependency>
>>> ... etc
>>> Can you please submit a patch for both modules?
>>> Thanks
>>> Lewis
>>> On Thu, Sep 10, 2015 at 1:25 PM, Mallder, Valerie <
>>> wrote:
>>>> Hi All,
>>>> Have a question for ya.  How in the world do y'all run the workflow
>>>> manager and resource manager right out of the box?  I mean, aside from
>>>> having to configure the properties files and policy files, how do you
>>>> get by with just what's in their lib directories?
>>>> When I run a simple system consisting of filemgr and workflow manager
>>>> and PGE tasks, the workflow manager crashes because it is unable to
>>>> load the following classes PGETaskInstance (from cas-pge-0.x.jar),
>>>> ProductCrawler (from cas-crawler-0.x.jar) and CoreMetKeys (from
>>>> cas-filemgr-0.x.jar). None of these jar files are in the workflow/lib
>>>> directory by default. I always end up copying them to the
>>>>workflow/lib directory.
>>>> The same thing happens if I add the resource manager to the mix, the
>>>> resource manager crashes right off the bat because it can't load
>>>> TaskJobInput (from caas-workflow-0.x.jar), StdPGETaskInstance (from
>>>> cas-pge-0.x.jar), ProductCrawler (cas-crawler-0.x.jar) and CoreMetKeys
>>>> (from cas-filemgr-0.x.jar). None of these jars are in the resource/lib
>>>> directory by default, and I have to copy them over.
>>>> How can this be?  Am I the only one this has happened to?
>>>> I opened a JIRA issue for this but then I thought I better ask whether
>>>> or not there is a valid reason why these jars are not included in
>>>> workflow/lib and resource/lib directories. And if there is, then does
>>>> that mean that everyone does the same thing I do and just copy these
>>>> jar files to workflow/lib and resource/lib?
>>>> Thanks!
>>>> Val
>>>> Valerie A. Mallder
>>>> New Horizons Deputy Mission System Engineer The Johns Hopkins
>>>> University/Applied Physics Laboratory
>>>> 11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
>>>> 240-228-7846 (Office) 410-504-2233 (Blackberry)
>>> --
>>> *Lewis*

View raw message