hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jimmy Xiang" <jxi...@cloudera.com>
Subject Re: Review Request 34293: HIVE-10721 SparkSessionManagerImpl leaks SparkSessions [Spark Branch]
Date Mon, 18 May 2015 03:25:17 GMT


> On May 18, 2015, 2:37 a.m., chengxiang li wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionManagerImpl.java,
line 88
> > <https://reviews.apache.org/r/34293/diff/1/?file=961679#file961679line88>
> >
> >     Just curious, it looks to me that AtomaticBoolean works here either, is that
possible 2 threads executed this block both?

If several sessions connect to the same HS2, they might execute this block concurrently. One
issue with AtomaticBoolean instead of synchonized here is that we have to make sure the SparkClientFactory
is properly initialized. Sometimes, we see it throws an exception, in which case, we may need
to initialize it again.


- Jimmy


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34293/#review84096
-----------------------------------------------------------


On May 15, 2015, 9:53 p.m., Jimmy Xiang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34293/
> -----------------------------------------------------------
> 
> (Updated May 15, 2015, 9:53 p.m.)
> 
> 
> Review request for hive and Xuefu Zhang.
> 
> 
> Bugs: HIVE-10721
>     https://issues.apache.org/jira/browse/HIVE-10721
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Add a SparkSession to createdSessions only after the session is opened properly if doOpen
is specified.
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java 7e33a3f

>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/RemoteHiveSparkClient.java bae30f3

>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java 603f1ca

>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionManagerImpl.java
ad012b6 
>   spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java 1bcd221

> 
> Diff: https://reviews.apache.org/r/34293/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jimmy Xiang
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message