hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao Sun (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15662) check startTime in SparkTask to make sure startTime is not less than submitTime
Date Fri, 20 Jan 2017 21:30:26 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832429#comment-15832429
] 

Chao Sun commented on HIVE-15662:
---------------------------------

+1. Maybe better to add a comment above the code to explain when that will happen. Thanks.

> check startTime in SparkTask to make sure startTime is not less than submitTime
> -------------------------------------------------------------------------------
>
>                 Key: HIVE-15662
>                 URL: https://issues.apache.org/jira/browse/HIVE-15662
>             Project: Hive
>          Issue Type: Bug
>            Reporter: zhihai xu
>            Assignee: zhihai xu
>            Priority: Minor
>         Attachments: HIVE-15662.000.patch
>
>
> Check startTime in SparkTask to make sure startTime is not less than submitTime. We saw
a corner case when the sparkTask is finished in less than 1 second, the startTime may not
be set because RemoteSparkJobMonitor will sleep for 1 second then check the state, in this
case, right after sleep for one second, the spark job is already completed. one example query
with 3 spark tasks, the second one finished quickly for around 1 second:
> {code}
> SparkTask1:
> "finishTime":1484638391978
> "submitTime":1484638385933
> "startTime": 1484638386973
> SparkTask2:
> "finishTime":1484638393019
> "submitTime":1484638391979
> "startTime": 1484638386973
> SparkTask3:
> "finishTime":1484638432123
> "submitTime":1484638393020
> "startTime": 1484638394057
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message