livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <van...@cloudera.com>
Subject Re: Can not deserialize instance of scala.collection.immutable.List out of VALUE_STRING (through reference chain: org.apache.livy.server.interactive.CreateInteractiveRequest)
Date Mon, 29 Jan 2018 23:37:41 GMT
"jars" is a list so you should probably try:

 "jars": [ "hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar" ]

On Mon, Jan 29, 2018 at 3:35 PM, Kyunam Kim <kim.kyunam@gmail.com> wrote:
> I get the following when I try "jars" in POST /sessions.
>
> I tried:
> - "jars": "hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar"
> - "jars": "/user/root/1x/my-lib-SNAPSHOT.jar"
>
> livy.spark.deploy-mode = cluster (client is also tried)
>
> Basically, I just want to load my custom library into LIVY.
>
> By the way, even if this works, do I need to let Spark know about my custom
> library by using "conf:
> {"spark.jars":"hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar"}" ??
>
>
> curl -k -v -H 'Content-Type: application/json' -X POST -d
> '{"kind":"spark","driverMemory":"1000M","executorCores":2,"jars":
> "hdfs://myserver:8020/user/root/1x/my-lib-SNAPSHOT.jar"}'
> "http://myserver:8998/sessions"
> Note: Unnecessary use of -X or --request, POST is already inferred.
> *   Trying 10.49.51.117...
> * TCP_NODELAY set
> * Connected to ... port 8998 (#0)
>> POST /sessions HTTP/1.1
>> Host: ...:8998
>> User-Agent: curl/7.55.1
>> Accept: */*
>> Content-Type: application/json
>> Content-Length: 139
>>
> * upload completely sent off: 139 out of 139 bytes
> < HTTP/1.1 400 Bad Request
> < Date: Mon, 29 Jan 2018 23:21:04 GMT
> < Content-Type: application/json; charset=UTF-8
> < Content-Length: 251
> < Server: Jetty(9.2.16.v20160414)
> <
> * Connection #0 to host myserver left intact
> "Can not deserialize instance of scala.collection.immutable.List out of
> VALUE_STRING token\n at [Source: HttpInputOverHTTP@1f8a778f; line: 1,
> column: 57] (through reference chain:
> org.apache.livy.server.interactive.CreateInteractiveRequest[\"jars\"])"
>
> Thanks,
> Q



-- 
Marcelo

Mime
View raw message