hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gour Saha (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-4793) [Umbrella] Simplified API layer for services and beyond
Date Sat, 04 Jun 2016 02:15:59 GMT

     [ https://issues.apache.org/jira/browse/YARN-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Gour Saha updated YARN-4793:
    Attachment: 20160603-YARN-Simplified-V1-API-Examples.adoc

Attached is the first pdf draft (20160603) of the higher order API specification for YARN

An HTML version is also available here - 

Thanks to [~vinodkv], [~sidharta-s], [~vvasudev], [~billie.rinaldi] and [~kkaranasos] for
contributing to and/or performing extensive reviews of the specification.

The source of the specification is in Swagger format and JSON examples are in asciidoc format
(both uploaded to this JIRA). I used the [swagger2markup-cli|https://github.com/Swagger2Markup/swagger2markup-cli]
utility to auto generate the pdf and html.

Few points to keep in mind while reviewing the document -
- Application name is a unique identifier for a specific user. It is more user-friendly and
easy to remember than the YARN application identifier.
- Few of the advanced features referred to in the spec is still in the works like – Placement
Policy (YARN-4902), Cluster/Docker networking (YARN-4007)
- Readiness check for containers and service needs architectural proof of concept

I have an early prototype of the REST API Service, implemented against this API spec. It is
built on top of Apache Slider and uses slider-core as a dependent module. If there is consensus
on the spec, I can submit a patch.

> [Umbrella] Simplified API layer for services and beyond
> -------------------------------------------------------
>                 Key: YARN-4793
>                 URL: https://issues.apache.org/jira/browse/YARN-4793
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Gour Saha
>         Attachments: 20160603-YARN-Simplified-V1-API-Examples.adoc, 20160603-YARN-Simplified-V1-API-Layer-For-Services.pdf,
> [See overview doc at YARN-4692, modifying and copy-pasting some of the relevant pieces
and sub-section 3.3.2 to track the specific sub-item.]
> Bringing a new service on YARN today is not a simple experience. The APIs of existing
frameworks are either too low­ level (native YARN), require writing new code (for frameworks
with programmatic APIs ) or writing a complex spec (for declarative frameworks).
> In addition to building critical building blocks inside YARN (as part of other efforts
at YARN-4692), we should also look to simplifying the user facing story for building services.
Experience of projects like Slider building real-­life services like HBase, Storm, Accumulo,
Solr etc gives us some very good learnings on how simplified APIs for building services will
look like.
> To this end, we should look at a new simple-services API layer backed by REST interfaces.
The REST layer can act as a single point of entry for creation and lifecycle management of
YARN services. Services here can range from simple single-­component apps to the most complex,
multi­-component applications needing special orchestration needs.
> We should also look at making this a unified REST based entry point for other important
features like resource­-profile management (YARN-3926), package-definitions' lifecycle­-management
and service­-discovery (YARN-913 / YARN-4757). We also need to flesh out its relation to
our present much ­lower level REST APIs (YARN-1695) in YARN for application-­submission
and management.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message