hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4692) [Umbrella] Simplified and first-class support for services in YARN
Date Thu, 18 Feb 2016 18:03:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-4692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15152735#comment-15152735

Arun Suresh commented on YARN-4692:

[~leftnoteasy], Am not entirely convinced that Priorities will handle every case.

Currently, a priority dictates the order in which the scheduler serves allocations to a AMs
resource request. Assuming a service is composed of containers of 3 types (Slider i think
currently calls them *roles* and each are mapped to a yarn priority). 

Taking the example of HBase, with Roles = Zookeeper, HBaseMaster and HBaseRegionServer in
decreasing priority order. This ensures that zookeeper containers are started first.. and
HBaseRS are started last. This establishes a dependency order between Roles. Now I would argue
that the *Preemptability* cost of an HBaseRS container is higher than ZK, since, ZK can still
function without any problems if 1 or 2 container goes down (depending on the size of the
quorum) but if an RS goes down, it might take a while for the regions hosted by the RS to
be available again.

> [Umbrella] Simplified and first-class support for services in YARN
> ------------------------------------------------------------------
>                 Key: YARN-4692
>                 URL: https://issues.apache.org/jira/browse/YARN-4692
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>         Attachments: YARN-First-Class-And-Simplified-Support-For-Services-v0.pdf
> YARN-896 focused on getting the ball rolling on the support for services (long running
applications) on YARN.
> I’d like propose the next stage of this effort: _Simplified and first-class support
for services in YARN_.
> The chief rationale for filing a separate new JIRA is threefold:
>  - Do a fresh survey of all the things that are already implemented in the project
>  - Weave a comprehensive story around what we further need and attempt to rally the community
around a concrete end-goal, and
>  - Additionally focus on functionality that YARN-896 and friends left for higher layers
to take care of and see how much of that is better integrated into the YARN platform itself.

This message was sent by Atlassian JIRA

View raw message