hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hitesh Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1390) Provide a way to capture source of an application to be queried through REST or Java Client APIs
Date Mon, 11 Nov 2013 21:28:17 GMT

    [ https://issues.apache.org/jira/browse/YARN-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13819416#comment-13819416

Hitesh Shah commented on YARN-1390:

bq. If we allow setting multiple applicationTypes (or applicationSources) and by default add
to the list, this is implicitly addressed. Between having a single applicationType field (with
potentially multiple values) to address both requirements or a separate field, there might
not be much difference. Do you see any drawbacks of using applicationType for these multiple
values? Thanks.

Yes - I believe there is quite a bit of difference. How is this list of application types
meant to be interpreted and by whom? Who defines ( and enforces ) the serialized structure
of this list? ApplicationType supporting the single application framework type is very well
defined and can be used by multiple components within YARN. How do you expect someone to search
for all mapreduce jobs? Do a substring search? 

To continue on what Zhijie said, I think tags should probably be a different attribute on
an application in addition to the applicationType. Rules on tags ( such as absence/presence)
are not enforceable but an application should always have an applicationType. Of course, if
we are just discussing implementation level details, applicationType could easily be implemented
in a generic way via a notion of a tag or more appropriately an attribute with a value. 

> Provide a way to capture source of an application to be queried through REST or Java
Client APIs
> ------------------------------------------------------------------------------------------------
>                 Key: YARN-1390
>                 URL: https://issues.apache.org/jira/browse/YARN-1390
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: api
>    Affects Versions: 2.2.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
> In addition to other fields like application-type (added in YARN-563), it is useful to
have an applicationSource field to track the source of an application. The application source
can be useful in (1) fetching only those applications a user is interested in, (2) potentially
adding source-specific optimizations in the future. 
> Examples of sources are: User-defined project names, Pig, Hive, Oozie, Sqoop etc.

This message was sent by Atlassian JIRA

View raw message