hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oleksandr Shevchenko (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-9023) Pluggable YARN Auxiliary Services
Date Thu, 15 Nov 2018 09:43:00 GMT
Oleksandr Shevchenko created YARN-9023:

             Summary: Pluggable YARN Auxiliary Services
                 Key: YARN-9023
                 URL: https://issues.apache.org/jira/browse/YARN-9023
             Project: Hadoop YARN
          Issue Type: New Feature
            Reporter: Oleksandr Shevchenko
            Assignee: Oleksandr Shevchenko
         Attachments: Pluggable YARN Auxiliary Services Doc v1.pdf

The propose is to create customizable auxiliary services infrastructure for all YARN services
to provide abilities to extends or change functionality for all YARN demons.
Currently, YARN has the ability to add auxiliary services only for NodeManager.

There are some cases when we need to add auxiliary services for some other YARN demons. For
example, in YARN-2267 proposed to add Auxiliary Service support in RM for implementation monitor
plugins or some alarm services. We faced with one more case, we need to create/setup some
storage for YARN demon data (for example for RM state store or AM intermediate data) before
demon started. Also, will be good if we can use some auxiliary service in different YARN demons.

The end users can use YARN Auxiliary Services (YAS) to customize or extend the functionality
of all YARN demons. We can provide an ability to create aux service both for one and for all
demons. This will make easy creating of aux service for shuffle or just to up some service
within another YARN daemon. We will get a unified and more extensible architecture for all
aux services which may need to create in the future.

Please look at the attached design doc for the details. [^Pluggable YARN Auxiliary Services
Doc v1.pdf]
Also, you can add your comments to this doc: https://docs.google.com/document/d/1Pxs8eeWZwnlwryPj7gyN8Ej1O-FRBSX_C0ispV9ORT8/edit?usp=sharing
Thanks a lot for any comments and attention to this feature!

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message