hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang" <xzh...@cloudera.com>
Subject Re: Review Request 29954: HIVE-9179. Add listener API to JobHandle.
Date Fri, 16 Jan 2015 23:15:43 GMT


> On Jan. 16, 2015, 10:35 p.m., Xuefu Zhang wrote:
> > spark-client/src/main/java/org/apache/hive/spark/client/JobHandleImpl.java, line
179
> > <https://reviews.apache.org/r/29954/diff/1-2/?file=823286#file823286line179>
> >
> >     Here sparkJobIds.add() is in the synchronized block. However, we have code accessing
the same variable (sparkJobIds) such as in RemoteSparkJobStatus class. Does that also needs
protection?
> 
> Marcelo Vanzin wrote:
>     No, we don't. The job id list itself is thread-safe. The synchronization happens
here so that we notify all listeners of everything. We don't want a listener being registered
concurrently with a new spark job arriving to miss that event.
>     
>     (That reminds me that I probably should switch the order of events around if a listener
is added after the handle is in a final state. Stay tuned.)

In that case, can we move sparkJobIds.add() outside the sync block?


- Xuefu


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29954/#review68492
-----------------------------------------------------------


On Jan. 16, 2015, 9:22 p.m., Marcelo Vanzin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29954/
> -----------------------------------------------------------
> 
> (Updated Jan. 16, 2015, 9:22 p.m.)
> 
> 
> Review request for hive, Brock Noland, chengxiang li, and Xuefu Zhang.
> 
> 
> Bugs: HIVE-9179
>     https://issues.apache.org/jira/browse/HIVE-9179
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-9179. Add listener API to JobHandle.
> 
> 
> Diffs
> -----
> 
>   spark-client/pom.xml 77016df61a0bcbd94058bcbd2825c6c210a70e14 
>   spark-client/src/main/java/org/apache/hive/spark/client/BaseProtocol.java f9c10b196ab47b5b4f4c0126ad455869ab68f0ca

>   spark-client/src/main/java/org/apache/hive/spark/client/JobHandle.java e760ce35d92bedf4d301b08ec57d1c2dc37a39f0

>   spark-client/src/main/java/org/apache/hive/spark/client/JobHandleImpl.java 1b8feedb0b23aa7897dc6ac37ea5c0209e71d573

>   spark-client/src/main/java/org/apache/hive/spark/client/RemoteDriver.java 0d49ed3d9e33ca08d6a7526c1c434a0dd0a06a67

>   spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java a30d8cbbaae9d25b1cffdc286b546f549e439545

>   spark-client/src/test/java/org/apache/hive/spark/client/TestJobHandle.java PRE-CREATION

>   spark-client/src/test/java/org/apache/hive/spark/client/TestSparkClient.java 795d62c776cec5e9da2a24b7d40bc749a03186ab

> 
> Diff: https://reviews.apache.org/r/29954/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Marcelo Vanzin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message