hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-7217) Improve API service usability for updating service spec and state
Date Thu, 19 Oct 2017 15:58:01 GMT

     [ https://issues.apache.org/jira/browse/YARN-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eric Yang updated YARN-7217:
----------------------------
    Description: 
API service for deploy, and manage YARN services have several limitations.

The update service API provides multiple functions:

# Stopping a service.
# Start a service.
# Increase or decrease number of containers.

The overloading is buggy depending on how the configuration should be applied.

Scenario 1
A user retrieves Service object from getService call, and the Service object contains state:
STARTED.  The user would like to increase number of containers for the deployed service. 
The JSON has been updated to increase container count.  The PUT method does not actually increase
container count.

Scenario 2
A user retrieves Service object from getService call, and the Service object contains state:
STOPPED.  The user would like to make a environment configuration change.  The configuration
does not get updated after PUT method.

This is possible to address by rearranging the logic of START/STOP after configuration update.
 However, there are other potential combinations that can break PUT method.  For example,
user like to make configuration changes, but not yet restart the service until a later time.

The alternative is to separate the PUT method into PUT method for configuration vs status.
 This increase the number of action that can be performed.  New API could look like:

{code}
@PUT
/ws/v1/services/[service_name]/spec

Request Data:
{
  "name":"[service_name]",
  "number_of_containers": 5
}
{code}

{code}
@PUT
/ws/v1/services/[service_name]/state

Request data:
{
  "name": "[service_name]",
  "state": "STOPPED|STARTED"
}
{code}

Scenario 3
There is no API to list all deployed applications by the same user.

Scenario 4
Desired state (spec) and current state are represented by the same Service object.  There
is no easy way to identify "state" is desired state to reach or, the current state of the
service.  It would be nice to have ability to retrieve both desired state, and current state
with separated entry points.  By implementing /spec and /state, it can resolve this problem.

Scenario 5
List all services deploy by the same user will trigger a directory listing operation on namenode.
 When hundred of users use Service UI to view or deploy applications, this will trigger denial
of services attack on namenode.  Recommendation is to have a cache layer for storing service
metadata.

  was:
The PUT method for updateService API provides multiple functions:

# Stopping a service.
# Start a service.
# Increase or decrease number of containers.

The overloading is buggy depending on how the configuration should be applied.

Scenario 1
A user retrieves Service object from getService call, and the Service object contains state:
STARTED.  The user would like to increase number of containers for the deployed service. 
The JSON has been updated to increase container count.  The PUT method does not actually increase
container count.

Scenario 2
A user retrieves Service object from getService call, and the Service object contains state:
STOPPED.  The user would like to make a environment configuration change.  The configuration
does not get updated after PUT method.

This is possible to address by rearranging the logic of START/STOP after configuration update.
 However, there are other potential combinations that can break PUT method.  For example,
user like to make configuration changes, but not yet restart the service until a later time.

The alternative is to separate the PUT method into PUT method for configuration vs status.
 This increase the number of action that can be performed.  New API could look like:

{code}
@PUT
/ws/v1/services/[service_name]/config

Request Data:
{
  "name":"[service_name]",
  "number_of_containers": 5
}
{code}

{code}
@PUT
/ws/v1/services/[service_name]/state

Request data:
{
  "name": "[service_name]",
  "state": "STOPPED|STARTED"
}
{code}


> Improve API service usability for updating service spec and state
> -----------------------------------------------------------------
>
>                 Key: YARN-7217
>                 URL: https://issues.apache.org/jira/browse/YARN-7217
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, applications
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>         Attachments: YARN-7217.yarn-native-services.001.patch, YARN-7217.yarn-native-services.002.patch,
YARN-7217.yarn-native-services.003.patch, YARN-7217.yarn-native-services.004.patch
>
>
> API service for deploy, and manage YARN services have several limitations.
> The update service API provides multiple functions:
> # Stopping a service.
> # Start a service.
> # Increase or decrease number of containers.
> The overloading is buggy depending on how the configuration should be applied.
> Scenario 1
> A user retrieves Service object from getService call, and the Service object contains
state: STARTED.  The user would like to increase number of containers for the deployed service.
 The JSON has been updated to increase container count.  The PUT method does not actually
increase container count.
> Scenario 2
> A user retrieves Service object from getService call, and the Service object contains
state: STOPPED.  The user would like to make a environment configuration change.  The configuration
does not get updated after PUT method.
> This is possible to address by rearranging the logic of START/STOP after configuration
update.  However, there are other potential combinations that can break PUT method.  For example,
user like to make configuration changes, but not yet restart the service until a later time.
> The alternative is to separate the PUT method into PUT method for configuration vs status.
 This increase the number of action that can be performed.  New API could look like:
> {code}
> @PUT
> /ws/v1/services/[service_name]/spec
> Request Data:
> {
>   "name":"[service_name]",
>   "number_of_containers": 5
> }
> {code}
> {code}
> @PUT
> /ws/v1/services/[service_name]/state
> Request data:
> {
>   "name": "[service_name]",
>   "state": "STOPPED|STARTED"
> }
> {code}
> Scenario 3
> There is no API to list all deployed applications by the same user.
> Scenario 4
> Desired state (spec) and current state are represented by the same Service object.  There
is no easy way to identify "state" is desired state to reach or, the current state of the
service.  It would be nice to have ability to retrieve both desired state, and current state
with separated entry points.  By implementing /spec and /state, it can resolve this problem.
> Scenario 5
> List all services deploy by the same user will trigger a directory listing operation
on namenode.  When hundred of users use Service UI to view or deploy applications, this will
trigger denial of services attack on namenode.  Recommendation is to have a cache layer for
storing service metadata.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message