livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Graham Hukill <ghuk...@gmail.com>
Subject Re: user defined sessionId / URI for Livy sessions
Date Wed, 06 Sep 2017 13:26:01 GMT
That is ideal behavior for our purposes, thanks so much all.

-Graham

On Wed, Sep 6, 2017 at 8:52 AM, Saisai Shao <sai.sai.shao@gmail.com> wrote:

> Session recovery should be enabled by configuration. If it is enabled,
> session id will be recorded in the persistent storage, and the number will
> continue to increment after restart.
>
> On Wed, Sep 6, 2017 at 8:27 PM, Graham Hukill <ghukill@gmail.com> wrote:
>
>> Ah, also very good to know, thanks.  That might be a great option.
>>
>> I'm going to look into this today, but I'm curious, are sessions
>> recovered automatically?  or, if not, and a session is not recovered, does
>> the numbering continue to increment for sessionId's?
>>
>> Thanks,
>> Graham
>>
>> On Tue, Sep 5, 2017 at 9:19 PM, Vivek <viveksuvarna@yahoo.co.in> wrote:
>>
>>> One other option is to turn on session recovery.
>>> This ensures that the session ids are unique
>>>
>>> Sent from my iPhone
>>>
>>> On 6 Sep 2017, at 12:10 AM, Graham Hukill <ghukill@gmail.com> wrote:
>>>
>>> Prabhu,
>>>
>>> Thanks so much for the information, nice to know it's been formally
>>> requested.
>>>
>>> thanks,
>>> Graham
>>>
>>> On Tue, Sep 5, 2017 at 11:22 AM, Prabhu Kasinathan <
>>> vasuramprabhu@gmail.com> wrote:
>>>
>>>> Livy currently doesn't have this feature now.
>>>>
>>>> Here, is the Jira for that: https://issues.cloudera.
>>>> org/plugins/servlet/mobile#issue/LIVY-41. PayPal is already having
>>>> that feature in production and Meisam had submitted pull request for this
>>>> feature to Livy.
>>>>
>>>> https://github.com/meisam/livy/wiki/Design-doc-for-Livy-41:-
>>>> Accessing-sessions-by-name
>>>>
>>>>
>>>> Thanks*!*
>>>> Prabhu
>>>> iPhone. iTypos. iApologize.
>>>>
>>>> On Sep 5, 2017, at 7:51 AM, Graham Hukill <ghukill@gmail.com> wrote:
>>>>
>>>> I wondered if it was possible to set the session URI when creating a
>>>> new one?  I see from the Livy REST API documentation (
>>>> https://livy.incubator.apache.org/docs/latest/rest-api.html), under
>>>> POST /sessions, some of the arguments that you can pass when initializing
a
>>>> new session, such as "kind", "name", etc.
>>>>
>>>> As I understand it now, sessionId's are automatically created when
>>>> POSTing to /sessions, setting the ID as an incrementing digit from "0" when
>>>> Livy starts.  If Livy is restarted, the numbering drops back to "0".
>>>>
>>>> Normally this would not present a problem, but we are planning on
>>>> starting/stopping Livy sessions from another application, and could
>>>> envision a scenario where:
>>>>
>>>>    - a session is started in Livy at /sessions/0
>>>>    - that session ID is associated in our ancillary app with a
>>>>    particular user or job
>>>>    - Livy restarts, then when a new session is started, the pointer to
>>>>    /sessions/0 is actually a different Livy/SparkSession
>>>>
>>>> Ideally, it would be helpful if we could mint a session ID with a PUT
>>>> request, something like PUT /sessions/foobar, where "foobar" is the newly
>>>> created sessionId.
>>>>
>>>> Is this currently possible in any way?  or are there undocumented
>>>> configurations that can be passed with the POST request for creating new
>>>> sessions that would define the sessionId?
>>>>
>>>> thanks,
>>>> Graham
>>>>
>>>>
>>>
>>
>

Mime
View raw message