hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Kanter (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3040) [Data Model] Implement client-side API for handling flows
Date Wed, 18 Feb 2015 23:17:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326716#comment-14326716

Robert Kanter commented on YARN-3040:

# I think the Entities (YARN-3041) are mainly for writing/reading to/from the ATS store. 
Most of the information stored in those Entities are not needed by the user when submitting
a job.  All the user really needs to set is the IDs, and some of these we can make optional
or determine automatically (e.g. it's obvious which cluster it's running on)
# 100 characters per tag seems like it should be enough; if not, we can maybe increase this
limit?  It is marked as {{@Evolving}}
# Like other properties, we can add a method to JobClient or one of those classes that sets
the property.  For example, {{setFlowId(String id)}} would simply set the tag

Flows and related constructs don't currently exist in YARN.  Unless we add these as first-class
concepts to the rest of YARN outside of the ATS (e.g. instead of only being able to submit
YARN applications, you can also submit YARN flows; though this is looking more like Oozie...),
I think tags are the only way to track this information.

> [Data Model] Implement client-side API for handling flows
> ---------------------------------------------------------
>                 Key: YARN-3040
>                 URL: https://issues.apache.org/jira/browse/YARN-3040
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Robert Kanter
> Per design in YARN-2928, implement client-side API for handling *flows*. Frameworks should
be able to define and pass in all attributes of flows and flow runs to YARN, and they should
be passed into ATS writers.
> YARN tags were discussed as a way to handle this piece of information.

This message was sent by Atlassian JIRA

View raw message