hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4692) [Umbrella] Simplified and first-class support for services in YARN
Date Sun, 17 Apr 2016 19:25:26 GMT

    [ https://issues.apache.org/jira/browse/YARN-4692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15244843#comment-15244843

Karthik Kambatla commented on YARN-4692:

(Discussed this offline with [~vinodkv]. Posting here so others are in the loop and provide
their thoughts.)

Thanks for the quite comprehensive survey and putting the doc together, Vinod.

I would like for us to define what exactly we mean by services. I see three kinds:
# Long-running applications like streaming applications that run as a single user
# Services like HBase that run as a single user, but serve other users. These services might
want to use Yarn for deployment and fault-tolerance, but not necessarily for sharing resources
among its users or accounting their resource usage.
# Services like Impala/LLAP/Drill that serve other users and would like to use Yarn for sharing/accounting
resources among them. 

The document covers several requirements of the first two cases. I would like for us to understand
the requirements of the third case as well - e.g. requesting resources on behalf of other
users, delegating resources, latency of allocation etc. 

> [Umbrella] Simplified and first-class support for services in YARN
> ------------------------------------------------------------------
>                 Key: YARN-4692
>                 URL: https://issues.apache.org/jira/browse/YARN-4692
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>         Attachments: YARN-First-Class-And-Simplified-Support-For-Services-v0.pdf
> YARN-896 focused on getting the ball rolling on the support for services (long running
applications) on YARN.
> I’d like propose the next stage of this effort: _Simplified and first-class support
for services in YARN_.
> The chief rationale for filing a separate new JIRA is threefold:
>  - Do a fresh survey of all the things that are already implemented in the project
>  - Weave a comprehensive story around what we further need and attempt to rally the community
around a concrete end-goal, and
>  - Additionally focus on functionality that YARN-896 and friends left for higher layers
to take care of and see how much of that is better integrated into the YARN platform itself.

This message was sent by Atlassian JIRA

View raw message