hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lantao Jin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6280) Add a query parameter in ResourceManager Cluster Applications REST API to control whether or not returns ResourceRequest
Date Mon, 10 Apr 2017 09:56:42 GMT

    [ https://issues.apache.org/jira/browse/YARN-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962671#comment-15962671

Lantao Jin commented on YARN-6280:

[~sunilg] Seems this unit test failure also can reproduce in a clean working directory:
Failed tests: 
  TestRMRestart.testFinishedAppRemovalAfterRMRestart:1673 expected null, but was:<submit_time:
1491818079324 application_submission_context { application_id { id: 1 cluster_timestamp: 1491818079310
} application_name: "" queue: "default" priority { priority: 0 } am_container_spec { } cancel_tokens_when_complete:
true maxAppAttempts: 2 resource { memory: 1024 virtual_cores: 1 } applicationType: "YARN"
keep_containers_across_application_attempts: false attempt_failures_validity_interval: 0 am_container_resource_request
{ priority { priority: 0 } resource_name: "*" capability { memory: 1024 virtual_cores: 1 }
num_containers: 1 relax_locality: true node_label_expression: "" execution_type_request {
execution_type: GUARANTEED enforce_execution_type: false } } } user: "lajin" start_time: 1491818079325
application_state: RMAPP_FINISHED finish_time: 1491818079360>

Tests run: 30, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:33 min
[INFO] Finished at: 2017-04-10T02:54:39-07:00
[INFO] Final Memory: 60M/1073M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test)
on project hadoop-yarn-server-resourcemanager: There are test failures.
[ERROR] Please refer to /home/lajin/git/apache/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire-reports
for the individual test results.
[ERROR] -> [Help 1]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] For more information about the errors and possible solutions, please read the following
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[lajin@apache-3913 hadoop-yarn-server-resourcemanager]$ git status
# On branch trunk
nothing to commit, working directory clean
[lajin@apache-3913 hadoop-yarn-server-resourcemanager]$ 

> Add a query parameter in ResourceManager Cluster Applications REST API to control whether
or not returns ResourceRequest
> ------------------------------------------------------------------------------------------------------------------------
>                 Key: YARN-6280
>                 URL: https://issues.apache.org/jira/browse/YARN-6280
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager, restapi
>    Affects Versions: 2.7.3
>            Reporter: Lantao Jin
>            Assignee: Lantao Jin
>         Attachments: YARN-6280.001.patch, YARN-6280.002.patch, YARN-6280.003.patch, YARN-6280.004.patch,
YARN-6280.005.patch, YARN-6280.006.patch, YARN-6280.007.patch
> Begin from v2.7, the ResourceManager Cluster Applications REST API returns   ResourceRequest
list. It's a very large construction in AppInfo.
> As a test, we use below URI to query only 2 results:
> http://<rm http address:port>/ws/v1/cluster/apps?states=running,accepted&limit=2
> The results are very different:
> ||Hadoop version|Total Character|Total Word|Total Lines|Size||
> |2.4.1|1192|	42|	42|	1.2 KB|
> |2.7.1|1222179|	48740|	48735|	1.21 MB|
> Most RESTful API requesters don't know about this after upgraded and their old queries
may cause ResourceManager more GC consuming and slower. Even if they know this but have no
idea to reduce the impact of ResourceManager except slow down their query frequency.
> The patch adding a query parameter "showResourceRequests" to help requesters who don't
need this information to reduce the overhead. In consideration of compatibility of interface,
the default value is true if they don't set the parameter, so the behaviour is the same as

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message