livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 王峰 <wangfengfight...@gmail.com>
Subject Re: Livy POST Sessions can not work with conf
Date Fri, 03 Nov 2017 10:00:48 GMT
Thank you for getting back to me so quickly, I tried use  "conf" :
{"spark.dynamicAllocation.enabled":"false","spark.shuffle.service.enabled":"false"}
, but it still did not work. I will have a try on 0.4 version

2017-11-03 17:54 GMT+08:00 Saisai Shao <sai.sai.shao@gmail.com>:

> I think it should be worked, can you please test with 0.4 version of Livy.
> Also "conf" should be a map of string key to string value.
>
> "conf" : {"spark.dynamicAllocation.enabled":"false","spark.shuffle.se
> rvice.enabled":"false"}
>
> Besides, please be aware in the current Livy we only tested on local and
> yarn mode, we don't guarantee the correct behavior using Mesos cluster
> manager.
>
> On Fri, Nov 3, 2017 at 5:36 PM, 王峰 <wangfengfighting@gmail.com> wrote:
>
>> Hello everyone , I have meet a problem about Livy-0.3 when I run `POST
>> /sessions` to create a new interactive spark session
>>
>>
>> here is the post body
>> ```Content-Type: application/json; charset=utf-8
>> {
>>   "kind" : "spark",
>>   "proxyUser" : "root",
>>   "executorMemory" : "4G",
>>   "executorCores": 4,
>>   "numExecutors" : 4,
>>    "conf" : {"spark.dynamicAllocation.enabled":false,"spark.shuffle.serv
>> ice.enabled":false}
>> }
>> ```
>>
>> However, I found that this Livy session  allocated all of resources in
>> Mesos UI as Pic shows
>>
>> [image: 内嵌图片 1]
>>
>> It seem like that `conf` did not worked but numExecutors , executorCores
>> and executorMemory worked well..
>>
>> please help me  thanks...
>>
>
>

Mime
View raw message