hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7512) Support service upgrade via YARN Service API and CLI
Date Thu, 15 Mar 2018 21:34:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16401140#comment-16401140
] 

Wangda Tan commented on YARN-7512:
----------------------------------

Thanks [~csingh] for working on this design, a couple of questions/comments.

1) Regarding to upgrade, from my understanding, it will be more valueable to upgrade only
for component's artifact and component's replicas.

For other part, since we have environment subsistution, it is possible that a change of global
environment could cause all dependencies upgraded. For example:
{code:java}
Service: 
  name: my-service.
  environments:
    env1: value1
    env2: value2
  components:
    component: 
      name: comp1
      launch_command: "python $env1 ..."
      artifact: $env2-x-y-z
    component:
      name: comp2
      launch_command: "app $env2 ..."
{code}
Change of $env1/$env2 could cause all component needs to be upgraded, I'm not sure if that
is desired state.

Talked to [~csingh], existing upgrade only supports artifact, and all other fields will be
silently ignored. Probably we should rename the "upgrade" to something like "set-image"?

2) Regarding to existing interactions between user and native services to do updates. IIUC,
this is workflow proposed in the design doc, plz correct me if I was wrong:

a. User upload a new upgrade spec to HDFS.
 b. User call upgrade service to read the new spec.
 c. User call upgrade component(s) to update components and instances.
 d. User manually set finalize or YARN will automatically set finalize of services.

First of all, I'm not sure if changing service state to UPGRADING could bring and benefits.
IMHO, it is confusing to let user manual finalize an upgrade but not all components are upgraded.

And a/b look like a burden to end user. If user needs to call c anyway, why we need a/b.

My proposal is to keep APIs to only upgrade *artifact* of component and instance of components:
{code:java}
 
Response upgradeComponentArtifact(ServiceName, ComponentName, NewArtifactName, [LowerIndexBound,
HigherIndexBound])
{code}
The {{[LowerIndexBound, HigherIndexBound]}} is useful when user want to verify the artifact
works instead of bring down all instances. By default we will upgrade all instances of the
component.

Once API server received this, this operation will be persisted, and component transferred
to UPGRADING. And once all requested instances upgraded, component's state will be updated
to READY again.

3) Several future requirement to consider: 
 a. What's the rollback story? It might be good to list history artifacts of instances and
timestamp. Existing design doesn't support rollback to a specific version.
 b. What's the AM restart story? From Chandni now AM doesn't persistent upgrade status of
containers. We may need to consider this.

Thoughts?

cc:[~gsaha]/[~billie.rinaldi].

> Support service upgrade via YARN Service API and CLI
> ----------------------------------------------------
>
>                 Key: YARN-7512
>                 URL: https://issues.apache.org/jira/browse/YARN-7512
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Gour Saha
>            Assignee: Chandni Singh
>            Priority: Major
>             Fix For: yarn-native-services
>
>         Attachments: _In-Place Upgrade of Long-Running Applications in YARN_v1.pdf, _In-Place
Upgrade of Long-Running Applications in YARN_v2.pdf
>
>
> YARN Service API and CLI needs to support service (and containers) upgrade in line with
what Slider supported in SLIDER-787 (http://slider.incubator.apache.org/docs/slider_specs/application_pkg_upgrade.html)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message