ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vitaly Brodetskyi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-19224) Perf: Host Checks response is way too big (320MB for 1200 nodes)
Date Fri, 16 Dec 2016 13:47:58 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-19224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vitaly Brodetskyi updated AMBARI-19224:
---------------------------------------
    Attachment: AMBARI-19224.patch

> Perf: Host Checks response is way too big (320MB for 1200 nodes)
> ----------------------------------------------------------------
>
>                 Key: AMBARI-19224
>                 URL: https://issues.apache.org/jira/browse/AMBARI-19224
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-agent
>    Affects Versions: 2.5.0
>            Reporter: Vitaly Brodetskyi
>            Assignee: Vitaly Brodetskyi
>            Priority: Critical
>             Fix For: 2.5.0
>
>         Attachments: AMBARI-19224.patch
>
>
> Currently, parsing of Host Checks after registration happens on UI, and on 1200 nodes
it grows to 320mb of data, which hard to download and then parse. We can move parsing and
aggregating data on the server, so on UI we will receive only required info.
> Request URL:
> {noformat}
>  /api/v1/requests/24?fields=tasks/Tasks,Requests/inputs,Requests/request_status
> {noformat}
> Part of response:
> {code:java}
>  "tasks" : [
>     {
>       "href" : "http://104.196.95.1:8080/api/v1/requests/24/tasks/31414",
>       "Tasks" : {
>         "attempt_cnt" : 1,
>         "command" : "ACTIONEXECUTE",
>         "command_detail" : "check_host ACTIONEXECUTE",
>         "end_time" : 1481047483941,
>         "error_log" : "/home/testvbrodetskyi-0001/var/lib/ambari-agent/data/errors-31414.txt",
>         "exit_code" : 0,
>         "host_name" : "testvbrodetskyi-0001",
>         "id" : 31414,
>         "output_log" : "/home/testvbrodetskyi-0001/var/lib/ambari-agent/data/output-31414.txt",
>         "request_id" : 24,
>         "role" : "check_host",
>         "stage_id" : 0,
>         "start_time" : 1481047457478,
>         "status" : "COMPLETED",
>         "stderr" : "None",
>         "stdout" : "2016-12-06 18:04:25,626 - Host checks started.\n2016-12-06 18:04:25,626
- Check execute list: host_resolution_check\n2016-12-06 18:04:25,626 - IP address forward
resolution check started.\n2016-12-06 18:04:43,192 - There were 1200 host(s) that could not
resolve to an IP address.\n2016-12-06 18:04:43,193 - IP address forward resolution check completed.\n2016-12-06
18:04:43,215 - Host checks completed.\n\nCommand completed successfully!\n",
>         "structured_out" : {
>           "host_resolution_check" : {
>             "exit_code" : 0,
>             "failed_count" : 1200,
>             "failures" : [
>               {
>                 "type" : "FORWARD_LOOKUP",
>                 "host" : "testvbrodetskyi-0601",
>                 "cause" : [
>                   -2,
>                   "Name or service not known"
>                 ]
>               },
>               {
>                 "type" : "FORWARD_LOOKUP",
>                 "host" : "testvbrodetskyi-0602",
>                 "cause" : [
>                   -2,
>                   "Name or service not known"
>                 ]
>               },
> {code}
> In case of 1200 for each host we have1200 objects in failures array so in result we have
1200x1200 objects as: 
> {code:java}
>   {
>                 "type" : "FORWARD_LOOKUP",
>                 "host" : "testvbrodetskyi-0602",
>                 "cause" : [
>                   -2,
>                   "Name or service not known"
>                 ]
>               },
> {code}
> However we on UI we parse only hosts' names, we can reduce response size by changing
failures to just array of host_names like:
> "failures" : ["testvbrodetskyi-0602", "testvbrodetskyi-0603", "testvbrodetskyi-0604",
...]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message