hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Denys Kuzmenko (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-20737) LocalHiveSparkClient and SparkSession race condition fix
Date Sat, 13 Oct 2018 10:04:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648315#comment-16648315
] 

Denys Kuzmenko edited comment on HIVE-20737 at 10/13/18 10:03 AM:
------------------------------------------------------------------

{code:java}
public void open(HiveConf conf) throws HiveException {
    closeLock.readLock().lock();
    try {  
        ...     
        hiveSparkClient = HiveSparkClientFactory.createHiveSparkClient(conf, sessionId,
            SessionState.get().getSessionId());     
        ...
    } finally {
        closeLock.readLock().unlock();
    }
}
{code}
Double-checked locking should be used, otherwise, it might happen, than inner logic of the
"open" method will be attempted multiple times, even when session is already open.


was (Author: dkuzmenko):
{code:java}
public void open(HiveConf conf) throws HiveException {
    closeLock.readLock().lock();
    try {  
        ...     
        hiveSparkClient = HiveSparkClientFactory.createHiveSparkClient(conf, sessionId,
            SessionState.get().getSessionId());     
        ...
    } finally {
        closeLock.readLock().unlock();
    }
}
{code}
Double-checked locking should be used, otherwise, it might happen, than inner logic of the
"open" method will be attempted multiple times and initiate several SparkContexts within
the same Spark session. LocalHiveSparkClient, "new SparkContext()" will throw exception "Only
one SparkContext may be running in this JVM".

 

> LocalHiveSparkClient and SparkSession race condition fix
> --------------------------------------------------------
>
>                 Key: HIVE-20737
>                 URL: https://issues.apache.org/jira/browse/HIVE-20737
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>            Reporter: Denys Kuzmenko
>            Priority: Major
>         Attachments: HIVE-20737.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message