livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saisai Shao <sai.sai.s...@gmail.com>
Subject Re: user defined sessionId / URI for Livy sessions
Date Wed, 06 Sep 2017 12:52:20 GMT
Session recovery should be enabled by configuration. If it is enabled,
session id will be recorded in the persistent storage, and the number will
continue to increment after restart.

On Wed, Sep 6, 2017 at 8:27 PM, Graham Hukill <ghukill@gmail.com> wrote:

> Ah, also very good to know, thanks.  That might be a great option.
>
> I'm going to look into this today, but I'm curious, are sessions recovered
> automatically?  or, if not, and a session is not recovered, does the
> numbering continue to increment for sessionId's?
>
> Thanks,
> Graham
>
> On Tue, Sep 5, 2017 at 9:19 PM, Vivek <viveksuvarna@yahoo.co.in> wrote:
>
>> One other option is to turn on session recovery.
>> This ensures that the session ids are unique
>>
>> Sent from my iPhone
>>
>> On 6 Sep 2017, at 12:10 AM, Graham Hukill <ghukill@gmail.com> wrote:
>>
>> Prabhu,
>>
>> Thanks so much for the information, nice to know it's been formally
>> requested.
>>
>> thanks,
>> Graham
>>
>> On Tue, Sep 5, 2017 at 11:22 AM, Prabhu Kasinathan <
>> vasuramprabhu@gmail.com> wrote:
>>
>>> Livy currently doesn't have this feature now.
>>>
>>> Here, is the Jira for that: https://issues.cloudera.
>>> org/plugins/servlet/mobile#issue/LIVY-41. PayPal is already having that
>>> feature in production and Meisam had submitted pull request for this
>>> feature to Livy.
>>>
>>> https://github.com/meisam/livy/wiki/Design-doc-for-Livy-41:-
>>> Accessing-sessions-by-name
>>>
>>>
>>> Thanks*!*
>>> Prabhu
>>> iPhone. iTypos. iApologize.
>>>
>>> On Sep 5, 2017, at 7:51 AM, Graham Hukill <ghukill@gmail.com> wrote:
>>>
>>> I wondered if it was possible to set the session URI when creating a new
>>> one?  I see from the Livy REST API documentation (
>>> https://livy.incubator.apache.org/docs/latest/rest-api.html), under
>>> POST /sessions, some of the arguments that you can pass when initializing a
>>> new session, such as "kind", "name", etc.
>>>
>>> As I understand it now, sessionId's are automatically created when
>>> POSTing to /sessions, setting the ID as an incrementing digit from "0" when
>>> Livy starts.  If Livy is restarted, the numbering drops back to "0".
>>>
>>> Normally this would not present a problem, but we are planning on
>>> starting/stopping Livy sessions from another application, and could
>>> envision a scenario where:
>>>
>>>    - a session is started in Livy at /sessions/0
>>>    - that session ID is associated in our ancillary app with a
>>>    particular user or job
>>>    - Livy restarts, then when a new session is started, the pointer to
>>>    /sessions/0 is actually a different Livy/SparkSession
>>>
>>> Ideally, it would be helpful if we could mint a session ID with a PUT
>>> request, something like PUT /sessions/foobar, where "foobar" is the newly
>>> created sessionId.
>>>
>>> Is this currently possible in any way?  or are there undocumented
>>> configurations that can be passed with the POST request for creating new
>>> sessions that would define the sessionId?
>>>
>>> thanks,
>>> Graham
>>>
>>>
>>
>

Mime
View raw message