hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Barna Zsombor Klara <>
Subject Re: Review Request 53166: HIVE-14910: Flaky test: TestSparkClient.testJobSubmission
Date Wed, 02 Nov 2016 09:35:31 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Nov. 2, 2016, 9:35 a.m.)

Review request for hive, Mohit Sabharwal, Siddharth Seth, and Xuefu Zhang.


Renamed the variables in JobHandleImp as requested in the review.

Repository: hive-git


HIVE-14910: Flaky test: TestSparkClient.testJobSubmission
I ran into this problem today while investigating a flaky test. I think the failure is coming
from this race condition: the listener can be added to the JobHandle only after the job has
been submitted. So there is no guarantee that every method of the listener will be invoked,
some state changes may have happened before the caller received the handler back.
I propose a slight change in the API. We should add the listeners as an argument of the submit
method, so we can set them on the Handler before the job itself is submitted. This way any
status change should be signalled to the listener.

Diffs (updated)

  spark-client/src/main/java/org/apache/hive/spark/client/ 44aa255a8271894ed3e787c3e7d1323628db63c4

  spark-client/src/main/java/org/apache/hive/spark/client/ 17c8f40edd472682d5604f41980d06e60cc92893

  spark-client/src/main/java/org/apache/hive/spark/client/ 3e921a5d9b77966d368684ee7b6f1c861ac60e08

  spark-client/src/main/java/org/apache/hive/spark/client/ e2a30a76e0f7fe95d8a453f502311baa08abcbe2

  spark-client/src/test/java/org/apache/hive/spark/client/ e8f352dce9f618573c2d79e9b8c59e19fad7298a

  spark-client/src/test/java/org/apache/hive/spark/client/ b95cd7a05d44224b53bf2cef9170146b8b2eb4a8



Unit tests modified and tested.
I also ran a simple query with HoS as the execution engine.


Barna Zsombor Klara

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message