hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-7606) Design SparkSession, SparkSessionManager
Date Mon, 04 Aug 2014 22:00:12 GMT

     [ https://issues.apache.org/jira/browse/HIVE-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Brock Noland updated HIVE-7606:
-------------------------------

    Description: 
In this JIRA we'll design two interfaces:

* SparkSession
* SparkSessionPoolManager

and then once that is agreed upon we'll design two implementations:

* SparkSessionImpl
* SparkSessionPoolManagerImpl

the form and function of these will be similar to the Tez equivalents. However, it appears
as though TezSession provides both the SessionState and the "client" Thus I think SparkSession
owns SparkClient but the implementation of what SparkClient does should stay there and not
bleed into SparkSession. The scope of this jira is just to create the shells and basic functionality.
The implementations in this jira should be able to:

* Share a SparkSessionImpl across queries
* Defining when a session can be re-used
* Take ownership of SparkContext objects (Note we can only have a single SC until SPARK-2243
is resolved)

  was:
In this JIRA we'll design two interfaces:

SparkSession
SparkSessionPoolManager

and then once that is agreed upon we'll design two implementations:

SparkSessionImpl
SparkSessionPoolManagerImpl

the form and function of these will be similar to the Tez equivalents. The scope of this jira
is just to create the shells and basic functionality.


> Design SparkSession, SparkSessionManager
> ----------------------------------------
>
>                 Key: HIVE-7606
>                 URL: https://issues.apache.org/jira/browse/HIVE-7606
>             Project: Hive
>          Issue Type: Task
>            Reporter: Brock Noland
>
> In this JIRA we'll design two interfaces:
> * SparkSession
> * SparkSessionPoolManager
> and then once that is agreed upon we'll design two implementations:
> * SparkSessionImpl
> * SparkSessionPoolManagerImpl
> the form and function of these will be similar to the Tez equivalents. However, it appears
as though TezSession provides both the SessionState and the "client" Thus I think SparkSession
owns SparkClient but the implementation of what SparkClient does should stay there and not
bleed into SparkSession. The scope of this jira is just to create the shells and basic functionality.
The implementations in this jira should be able to:
> * Share a SparkSessionImpl across queries
> * Defining when a session can be re-used
> * Take ownership of SparkContext objects (Note we can only have a single SC until SPARK-2243
is resolved)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message