hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vihang Karajgaonkar (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-19514) Successful test logs should be copied
Date Sun, 13 May 2018 16:59:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vihang Karajgaonkar updated HIVE-19514:
---------------------------------------
    Description: 
Ptest tries to copy ~20G worth of logs from the worker nodes to the server. The total aggregate
time spent by all the Ptest server in copying these files is close to 15 min (although much
of that is in parallel threads). Most of the times we don't check for the logs. We seldom
check for logs of successful tests. We should ignore copying successful test logs by default
to improve runtime.

{noformat}
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:235 Executed 13568 tests
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase took 71 minutes
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase.TotalRsyncElapsedTime
took 15 minutes
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: PrepPhase took 5 minutes
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: ReportingPhase took 0 minutes
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: TestCheckPhase took 0 minutes
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: YetusPhase took 0 minutes
{noformat}

  was:
Ptest tries to copy ~20G worth of logs from the worker nodes to the server. The total aggregate
time spent by all the hosts in copying these files is close to 15 min in copying files. Most
of the times we don't check for the logs. We seldom check for logs of successful tests. We
should ignore copying successful test logs by default to improve runtime.

{noformat}
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:235 Executed 13568 tests
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase took 71 minutes
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase.TotalRsyncElapsedTime
took 15 minutes
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: PrepPhase took 5 minutes
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: ReportingPhase took 0 minutes
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: TestCheckPhase took 0 minutes
2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: YetusPhase took 0 minutes
{noformat}


> Successful test logs should be copied
> -------------------------------------
>
>                 Key: HIVE-19514
>                 URL: https://issues.apache.org/jira/browse/HIVE-19514
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Vihang Karajgaonkar
>            Assignee: Vihang Karajgaonkar
>            Priority: Major
>
> Ptest tries to copy ~20G worth of logs from the worker nodes to the server. The total
aggregate time spent by all the Ptest server in copying these files is close to 15 min (although
much of that is in parallel threads). Most of the times we don't check for the logs. We seldom
check for logs of successful tests. We should ignore copying successful test logs by default
to improve runtime.
> {noformat}
> 2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:235 Executed 13568 tests
> 2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase took
71 minutes
> 2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: ExecutionPhase.TotalRsyncElapsedTime
took 15 minutes
> 2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: PrepPhase took 5 minutes
> 2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: ReportingPhase took
0 minutes
> 2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: TestCheckPhase took
0 minutes
> 2018-05-13 16:10:15,243  INFO [TestExecutor] PTest.run:237 PERF: YetusPhase took 0 minutes
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message