livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: How do I set spark conf parameters?
Date Wed, 31 Jan 2018 10:45:15 GMT
Sorry. I found an answer online.It should be something like this

new LivyClientBuilder().setConf("spark.es.index.auto.create",
"true").setConf("spark.cassandra.connection.host",
"127.0.0.01").build();


On Wed, Jan 31, 2018 at 1:16 AM, kant kodali <kanth909@gmail.com> wrote:

> Hi All,
>
>
> How do I set Spark Conf Parameters ? The below doesnt seem to get picked
> up? If so, how can I change my program such that it can pick it up. I am
> not seeing a way if sparkcontext is already created?
>
> public String call(JobContext ctx) throws Exception {
>     ctx.sc().setLogLevel("INFO");
>     ctx.sc().getConf()
>         .set("spark.cassandra.connection.host", config.getString("cassandra.host"))
>         .set("spark.cassandra.auth.username", config.getString("cassandra.user"))
>         .set("spark.cassandra.auth.password", config.getString("cassandra.pass"))
>         .set("es.index.auto.create", "true");
>
> }
>
>
> Thanks!
>
>

Mime
View raw message