hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ray Chiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6142) Support rolling upgrade between 2.x and 3.x
Date Thu, 21 Sep 2017 23:41:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175663#comment-16175663
] 

Ray Chiang commented on YARN-6142:
----------------------------------

Security.proto
* Two new messages

yarn_protos.proto
* ResourceUtilizationProto
** YARN-4293.  2.8.0 and later.
* ContainerStateProto added 2 new enum
** YARN-4597.  2.8.0 and later.
* ContainerProto added 3 optional fields
** execution_type
*** YARN-5127.  2.9.0 and later.
** allocation_request_id
*** YARN-4887.  2.9.0 and later.
** version
*** YARN-5221.  2.8.0 and later.
* FinalApplicationStatusProto added 1 new enum
** YARN-4207.  2.8.0 and later.
* ApplicationResourceUsageReportProto added 4 optional fields
** queue_usage_percentage
*** YARN-4285.  2.8.0 and later.
** cluster_usage_percentage
*** YARN-4285.  2.8.0 and later.
** preempted_memory_seconds
*** YARN-4218.  2.8.0 and later.
** preempted_vcore_seconds
*** YARN-4218.  2.8.0 and later.
* ApplicationReportProto added 6 optional fields
** log_aggregation_status
*** YARN-1402.  2.8.0 and later.
** unmanaged_application
*** YARN-3543.  2.8.0 and later.
** priority
*** YARN-3948.  2.8.0 and later.
** appNodeLabelExpression
*** YARN-3717.  2.8.0 and later.
** amNodeLabelExpression
*** YARN-3717.  2.8.0 and later.
** appTimeouts
*** YARN-5965.  2.9.0 and later.
* New message AppTimeoutsMapProto
** YARN-5965.  2.9.0 and later.
* New message ApplicationTimeoutProto
** YARN-5965.  2.9.0 and later.
* New enum LogAggregationStatusProto
** YARN-1402.  2.8.0 and later.
* ApplicationAttemptReportProto added 2 optional fields
** start_time
*** YARN-3451.  2.8.0 and later.
** finish_time
*** YARN-3451.  2.8.0 and later.
* NodeStateProto added 2 new enums
** NS_DECOMMISSIONING
*** YARN-3225.  2.8.0 and later.
** NS_SHUTDOWN
*** YARN-41.  2.8.0 and later.
* NodeReportProto added 2 optional fields
** containers_utilization
*** YARN-4293.  2.8.0 and later.
** node_utilization
*** YARN-4293.  2.8.0 and later.
* New message NodeLabelProto
* New enum ContainerTypeProto
* New enum ExecutionTypeProto
* ResourceRequestProto added 2 optional fields
** execution_type_request
*** YARN-5180.  2.9.0 and later.
** allocation_request_id
*** YARN-4888.  2.9.0 and later.
* New message ExecutionTypeRequestProto
** YARN-4888.  2.9.0 and later.
* ApplicationSubmissionContextProto changed 1 field to repeated, added 1 optional field
** am_container_resource_request
*** YARN-6050.  2.9.0 and later.
** application_timeouts
*** YARN-4205.  2.9.0 and later.
* New enum ApplicationTimeoutTypeProto
** YARN-4205.  2.9.0 and later.
* New message ApplicationTimeoutMapProto
** YARN-4205.  2.9.0 and later.
* New message ApplicationUpdateTimeoutMapProto
** YARN-4205.  2.9.0 and later.
* LogAggregationContextProto added 2 optional fields
** log_aggregation_policy_class_name
*** YARN-221.  2.8.0 and later.
** log_aggregation_policy_parameters
*** YARN-221.  2.8.0 and later.
* YarnClusterMetricsProto added 5 optional fields
** All fields YARN-3348
* QueueStateProto added 1 new enum
** YARN-5756.  2.9.0 and later.
* New message QueueStatisticsProto
** YARN-3348.  2.8.0 and later.
* QueueInfoProto added 3 optional fields
** queueStatistics
** preemptionDisabled
** queueConfigurationsMap
* New message QueueConfigurationsProto
** YARN-6164.  2.9.0 and later.
* New message QueueConfigurationsMapRoto
** YARN-6164.  2.9.0 and later.
* New enum SignalContainerCommandProto
** YARN-1897.  2.8.0 and later.
* ReservationDefinitionProto added 2 optional fields
** recurrence_expression
*** YARN-5327.  2.9.0 and later.
** priority
*** YARN-5384.  2.9.0 and later.
* New message ResourceAllocationRequestProto
** YARN-4340.  2.8.0 and later.
* New message ReservationAllocationStateProto
** YARN-4340.  2.8.0 and later.
* ContainerLaunchContextProto added 2 optional fields
** container_retry_context
*** YARN-3998.  2.9.0 and later.
** tokens_conf
*** YARN-5910.  2.9.0 and later.
* ContainerStatusProto added 3 new optional fields
** capability
*** YARN-3866.  2.8.0 and later.
** executionType
*** YARN-2882.  2.9.0 and later.
** container_attributes
*** YARN-5430.  2.9.0 and later.
* Message ContainerResourceIncreaseRequestProto moved to yarn_service_protos.proto
** YARN-3866.  2.8.0 and later.
*** Still in 2.8.x
* Message ContainerResourceIncreaseProto moved to yarn_service_protos.proto
** YARN-3866.  2.8.0 and later.
*** Still in 2.8.x
* Message ContainerResourceDecreaseProto moved to yarn_service_protos.proto
** YARN-3866.  2.8.0 and later.
*** Still in 2.8.x
* New message ContainerRetryContextProto
** YARN-3998.  2.9.0 and later.

yarn_server_common_service_protos.proto
* New message RemoteNodeProto
* New message RegisterDistributedSchedulingAMResponseProto
* New message DistributedSchedulingAllocateResponseProto
* New message DistributedSchedulingAllocateRequestProto
* New message NodeLabelsProto
* RegisterNodeManagerRequestProto added 2 optional fields
* RegisterNodeManagerResponseProto added 2 optional fields
* New message UnRegisterNodeManagerRequestProto
* New message UnRegisterNodeManagerResponseProto
* NodeHeartbeatRequestProto added 3 optional fields
* New message message LogAggregationReportProto
* NodeHeartbeatResponseProto added 7 optional fields
* New message ContainerQueuingLimitProto
* New message AppCollectorsMapProto
* New message ReportNewCollectorInfoRequestProto
* New message ReportNewCollectorInfoResponseProto
* New message GetTimelineCollectorContextRequestProto
* New message GetTimelineCollectorContextResponseProto
* NMContainerStatusProto added 2 optional fields

yarn_server_resourcemanager_service_protos.proto
* RefreshNodesrequestProto added 2 optional fields
* AddToClusterNodeLabelsRequestProto deprecated field 1, added new field 2
* ReplaceLabelsOnNodeRequestProto added new field
* New message RefreshNodesResourcesRequestProto
* New message RefreshNodesResourcesResponseProto
* New message UpdateNodeLabelsResponseProto
* New message CheckForDecommissioningNodesRequestProto
* New message CheckForDecommissioningNodesResponseProto
* New message RefreshClusterMaxPriorityRequestProto
* New message RefreshClusterMaxPriorityResponseProto
* New message NodeIdToLabelsNameProto
* New enum DecommissionTypeProto

yarn_service_protos.proto
* New enum ContainerUpdateTypeProto
* New message UpdateContainerRequestProto
* New message UpdateContainerErrorProto
* AllocateRequestProto renamed ContainerResourceIncreaseRequestProto to UpdateContainerRequestProto
** Filed YARN-7219 to discuss
*** Initial version still in 2.8.x
* New message UpdatedContainerProto
* AllocateResponseProto removed 2 fields (ContainerResourceIncreaseProto, ContainerResourceDecreaseProto)
and added 4 optional fields
*** Initial version still in 2.8.x
* New message FailApplicationAttemptRequestProto
* New message FailApplicationAttemptResponseProto
* KillApplicationRequestProto added 1 optional field
* GetClusterNodeLabelsResponseProto deprecated field 1, added new field 2
* New message UpdateApplicationPriorityRequestProto
* New message UpdateApplicationPriorityResponseProto
* New message SignalContainerRequestProto
* New message SignalContainerResponseProto
* New message UpdateApplicationTimeoutsRequestProto
* New message UpdateApplicationTimeoutsResponseProto
* New message ResourceLocalizationRequestProto
* New message ResourceLocalizationResponseProto
* New message ReInitializeContainerRequestProto
* New message ReInitializeContainerResponseProto
* New message RestartContainerResponseProto
* New message RollbackResponseProto
* New message CommitResponseProto
* New message IncreaseContainersResourceRequestProto
* New message IncreaseContainersResourceResponseProto
* New message ContainerUpdateRequestProto
* New message ContainerUpdateResponseProto
* New message GetNewReservationRequestProto
* New message GetNewReservationResponseProto
* ReservationSubmissionRequestProto added 1 optional field (ReservationIdProto?)
* New message ReservationListRequestProto
* New message ReservationListResponseProto


> Support rolling upgrade between 2.x and 3.x
> -------------------------------------------
>
>                 Key: YARN-6142
>                 URL: https://issues.apache.org/jira/browse/YARN-6142
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: rolling upgrade
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Andrew Wang
>            Assignee: Ray Chiang
>            Priority: Blocker
>
> Counterpart JIRA to HDFS-11096. We need to:
> * examine YARN and MR's  JACC report for binary and source incompatibilities
> * run the [PB differ|https://issues.apache.org/jira/browse/HDFS-11096?focusedCommentId=15816405&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15816405]
that Sean wrote for HDFS-11096 for the YARN PBs.
> * sanity test some rolling upgrades between 2.x and 3.x. Ideally these are automated
and something we can run upstream.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message