oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: Obtaining workflow instance unique ID
Date Fri, 29 Apr 2011 04:52:01 GMT
Good point Bruce.

The ending event is probably important from a pedigree perspective or provenance perspective
too right (and also from Transactional I would bet).

Cheers,
Chris

On Apr 27, 2011, at 5:10 AM, Bruce Barkstrom wrote:

> Depends on which disciplinary basis you were to
> start with.  The one below is peculiar to a rather
> small community - although perhaps that's what
> you want if you're using it to describe the OODT
> environment for development.
> 
> In the broader industrial engineering context, a
> workflow is a graph describing jobs or activities.
> I've got at least one reference (Gershwin, Manufacturing
> Systems Engineering) that says every activity has
> two events - a starting event and an ending event.
> In this context, a Workflow might be thought of
> as roughly equivalent to a Work Breakdown Structure.
> 
> However, I suspect this is largely irrelevant to what
> OODT is doing.  The answer probably depends on
> which user community you're expecting to talk to.
> 
> Bruce R. Barkstrom
> 
> On Tue, Apr 26, 2011 at 11:56 PM, Cameron Goodale <sigep311@gmail.com> wrote:
>> Ok OODT List I am I think I am finally getting a formal OODT-WORKFLOW
>> Education here.  But I still have some questions related to Workflow in
>> general. I hope to get some understanding and to also NAIL DOWN the
>> vocabulary surrounding Workflow.  I read the excellent Developer Guide here:
>>  http://oodt.apache.org/components/maven/workflow/development/developer.html
>>  but I still have some question I hope I can get clarified.
>> 
>> Vocabulary (and a sort of TRUE/FALSE, so please correct the list I am
>> making)
>> 
>> 
>> Workflow Heirarchy:
>> 
>>   - Event
>>      - Workflow
>>         - Task = java class
>> 
>> 
>> Event
>> 
>>   - This is where [ALL] workflows must begin.
>>   - An Event initializes 1 or more workflows
>>   - IF an Event initializes more than 1 workflow the workflows will run in
>>   parallel
>>   - You cannot use standard configuration to Serialize Workflows spawned
>>   from a single Event
>>   - Events DO NOT generate an INSTANCE_ID
>> 
>> 
>> Workflow
>> 
>>   - Workflows DO generate an INSTANCE_ID
>>   - Workflows must be used to initialize tasks
>>   - Workflows are a way to logically GROUP Tasks together
>>   - Workflows spawn 1 or more tasks
>> 
>> Task
>> 
>>   - Lowest atomic level
>>   - Tasks map to a single Java Class
>>   - [Can tasks launch non-Java Code?]
>>   - [Can tasks be parallelized/serialized using config?]
>>   - [Do tasks generate an INSTANCE_ID?]
>>   - [Can tasks pass input/output to each other?]
>> 
>> 
>> I think that is enough questions/statements for now.  I am eager to read
>> your replies.
>> 
>> -Cameron
>> 
>> 
>> 
>> 
>> On Tue, Apr 26, 2011 at 4:47 AM, Bruce Barkstrom <brbarkstrom@gmail.com>wrote:
>> 
>>> As an outside thought, you might want to think of a workflow as
>>> a kind of graph whose vertices are either processes or "files" (metadata
>>> perhaps being one kind of file - if you want to think of database
>>> transactions
>>> as "files", you can add those to the category of "file").  The edges in the
>>> graph are events.  Maybe you want to have an inventory of vertices or
>>> an inventory of edges.  If you want to think about a hierarchical graph,
>>> that might be useful as well, although the data structure and access
>>> methods might be a bit more complex.
>>> 
>>> If you want a graph (or collection of graphs - one graph per workflow),
>>> you might use hashing on the ID's to access either vertices or edges.
>>> 
>>> Interesting problem.
>>> 
>>> Bruce R. Barkstrom
>>> 
>>> On Mon, Apr 25, 2011 at 11:00 PM, Verma, Rishi (317I)
>>> <Rishi.Verma@jpl.nasa.gov> wrote:
>>>> Hi Paul,
>>>> 
>>>> Thanks a lot for your suggestions.
>>>> 
>>>> I agree, the second approach is probably the way to go. Although, would
>>> it be more beneficial to save and return a single 'event' ID instead of
>>> multiple workflow instance IDs? I'm under the assumption that workflow
>>> behaves something akin to a job submission system. This assumption may be
>>> incorrect, but it might be beneficial to just get back a single unique ID
>>> for the launch of a workflow top-level event as opposed to getting back a
>>> list of IDs...
>>>> 
>>>> So perhaps within XmlRpcWorkflowManager.handleEvent, we could generate
>>> and save a unique ID for the single event? This ID could be queried by a
>>> client.
>>>> 
>>>> Thanks!
>>>> rishi
>>>> 
>>>> On Apr 25, 2011, at 1:25 PM, Ramirez, Paul M (388J) wrote:
>>>> 
>>>>> Hey All,
>>>>> 
>>>>> Thought this might be useful to the larger OODT community.
>>>>> 
>>>>> 
>>>>> 
>>>>> Hey Rishi,
>>>>> 
>>>>> I talked with Mike about this last week but never got back to him with
a
>>> concrete answer. However, after putting some more thought into it there are
>>> 2 approaches I can think of.
>>>>> 
>>>>> First Approach:
>>>>> 1) Put some type of user generated id in the metadata object that gets
>>> into the workflow instance metadata
>>>>> 2) Page through the lists of workflow instances that the workflow
>>> manager has and identify those with your tag. Remember an event can kick off
>>> more than one workflow.
>>>>> 3) Any that have metadata that match your user generated id are your
>>> workflow instances you're interested in.
>>>>> 
>>>>> Second Approach:
>>>>> 1) Add a new method to the client and server so that list of workflow
>>> instance ids is returned
>>>>> 
>>>>> Looking at XmlRpcWorkflowManager.java
>>>>> 
>>>>> 
>>>>>    public boolean handleEvent(String eventName, Hashtable metadata)
>>>>>            throws RepositoryException, EngineException {
>>>>>        LOG.log(Level.INFO<http://Level.INFO/>, "WorkflowManager:
>>> Received event: " + eventName);
>>>>> 
>>>>>        List workflows = null;
>>>>> 
>>>>>        try {
>>>>>            workflows = repo.getWorkflowsForEvent(eventName);
>>>>>        } catch (Exception e) {
>>>>>            e.printStackTrace();
>>>>>            throw new RepositoryException(
>>>>>                    "Exception getting workflows associated with event:
"
>>>>>                            + eventName + ": Message: " +
>>> e.getMessage());
>>>>>        }
>>>>> 
>>>>>        if (workflows != null) {
>>>>>            for (Iterator i = workflows.iterator(); i.hasNext();) {
>>>>>                Workflow w = (Workflow) i.next();
>>>>>                LOG.log(Level.INFO<http://Level.INFO/>,
>>> "WorkflowManager: Workflow " + w.getName()
>>>>>                        + " retrieved for event " + eventName);
>>>>> 
>>>>>                Metadata m = new Metadata();
>>>>>                m.addMetadata(metadata);
>>>>> 
>>>>>                try {
>>>>>                    engine.startWorkflow(w, m); // This returns a
>>> workflow instance which has an id that could be saved and returned
>>>>>                } catch (Exception e) {
>>>>>                    e.printStackTrace();
>>>>>                    throw new EngineException(
>>>>>                            "Engine exception when starting workflow:
"
>>>>>                                    + w.getName() + ": Message: "
>>>>>                                    + e.getMessage());
>>>>>                }
>>>>>            }
>>>>>            return true;
>>>>>        } else
>>>>>            return false;
>>>>>    }
>>>>> 
>>>>> This should probably be a new method on both the client and server and
>>> should be supplied as a patch. The method would return either List<String>
>>> or List<WorkflowInstance> probably the latter but maybe someone else will
>>> chime in. Of course this would have to have the accompanying client method
>>> but it shouldn't be that big of an update.
>>>>> 
>>>>> My instincts would say that second approach is probably the way to go.
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Paul
>>>>> 
>>>>> 
>>>>> 
>>>>> On Apr 25, 2011, at 11:39 AM, Verma, Rishi (317I) wrote:
>>>>> 
>>>>> 
>>>>>   return ((Boolean) client
>>>>>                    .execute("workflowmgr.handleEvent", argList))
>>>>>                    .booleanValue();
>>>>> 
>>>>> Is there a way to obtain and query for a workflow 'instance ID'?
>>>>> 
>>>>> Thanks!
>>>>> Rishi & Mike
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>> 
>> 
>> 
>> --
>> 
>> Sent from a Tin Can attached to a String
>> 


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Mime
View raw message