hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5079) [Umbrella] Native YARN framework layer for services and beyond
Date Thu, 26 May 2016 16:16:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302325#comment-15302325
] 

Vinod Kumar Vavilapalli commented on YARN-5079:
-----------------------------------------------

[~kasha]
bq. The mailing list might be more conducive for discussion. 
bq. How do we propose to accommodate Slider committers and PMC members, who are not committers/PMC
members on Yarn?
Agreed, I just wanted us to wrap up top-level technical discussions here and bring it to some
shape so that we can take the rest of the open issues community related discussions to the
mailing list.

[~gsaha]
{quote}
One way to do this, is by:
a. Create a branch in Hadoop and move Slider AM, Client and tests into it
b. Create a branch in Slider which will have only Agent and App-Packages and will point to
Slider core (AM and Client) as Hadoop modules
{quote}
This is the most practical way forward, +1.

bq. It is likely that we start, by contributing this code into a branch in Hadoop and then
work on an Agent-less or thin-Agent architecture. Subsequently, we can include this in one
of the early Hadoop-3 releases.
+1 again.

> [Umbrella] Native YARN framework layer for services and beyond
> --------------------------------------------------------------
>
>                 Key: YARN-5079
>                 URL: https://issues.apache.org/jira/browse/YARN-5079
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>
> (See overview doc at YARN-4692, modifying and copy-pasting some of the relevant pieces
and sub-section 3.3.1 to track the specific sub-item.)
> (This is a companion to YARN-4793 in our effort to simplify the entire story, but focusing
on APIs)
> So far, YARN by design has restricted itself to having a very low-­level API that can
support any type of application. Frameworks like Apache Hadoop MapReduce, Apache Tez, Apache
Spark, Apache REEF, Apache Twill, Apache Helix and others ended up exposing higher level APIs
that end­-users can directly leverage to build their applications on top of YARN. On the
services side, Apache Slider has done something similar.
> With our current attention on making services first­-class and simplified, it's time
to take a fresh look at how we can make Apache Hadoop YARN support services well out of the
box. Beyond the functionality that I outlined in the previous sections in the doc on how NodeManagers
can be enhanced to help services, the biggest missing piece is the framework itself. There
is a lot of very important functionality that a services' framework can own together with
YARN in executing services end­-to­-end.
> In this JIRA I propose we look at having a native Apache Hadoop framework for running
services natively on YARN.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message