flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Classloader error after SSL setup
Date Wed, 04 Oct 2017 12:11:24 GMT
The configuration looks reasonable. Just to be sure, are the paths 
accessible by all nodes?

As a first step, could you set the logging level to DEBUG (by modifying 
the 'conf/log4j.properties' file), resubmit the job (after a cluster 
restart) and check the Job- and TaskManager logs for any exception?

On 04.10.2017 03:15, Aniket Deshpande wrote:
> Background: We have a setup of Flink 1.3.1 along with a secure MAPR 
> cluster (Flink is running on mapr client nodes). We run this flink 
> cluster via flink-jobmanager.sh <http://flink-jobmanager.sh> 
> foreground and flink-taskmanager.sh <http://flink-taskmanager.sh> 
> foreground command via Marathon.  In order for us to make this work, 
> we had to add -Djavax.net 
> <http://-Djavax.net>.ssl.trustStore="$JAVA_HOME/jre/lib/security/cacerts" in 
> flink-console.sh <http://flink-console.sh> as extra JVM arg 
> (otherwise, flink was taking MAPR's ssl_truststore as default 
> truststore and then we were facing issues for any 3rd party jars like 
> aws_sdk etc.). This entire setup was working fine as it is and we 
> could submit our jars and the pipelines ran without any problem
>
>
> Problem: We started experimenting with enabling ssl for all 
> communication for Flink. For this, we followed 
> https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/security-ssl.html for

> generating CA and keystore. I added the following properties to 
> flink-conf.yaml:
>
>
> security.ssl.enabled: true
> security.ssl.keystore: /opt/flink/certs/node1.keystore
> security.ssl.keystore-password: <password>
> security.ssl.key-password: <password>
> security.ssl.truststore: /opt/flink/certs/ca.truststore
> security.ssl.truststore-password: <password>
> jobmanager.web.ssl.enabled: true
> taskmanager.data.ssl.enabled: true
> blob.service.ssl.enabled: true
> akka.ssl.enabled: true
>
>
> We then spin up a cluster and tried submitting the same job which was 
> working before. We get the following erros:
> org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot 
> load user class: 
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer09
> ClassLoader info: URL ClassLoader:
> Class not resolvable through given classloader.
> at 
> org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:229)

>
> at 
> org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:95)

>
> at 
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:230) 
>
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
> at java.lang.Thread.run(Thread.java:748)
>
>
> This error disappears when we remove the ssl config properties i.e run 
> flink cluster without ssl enabled.
>
>
> So, did we miss any steps for enabling ssl?
>
>
> P.S.: We tried removing the extra JVm arg mentioned above, but still 
> get the same error.
>
> -- 
>
> Aniket



Mime
View raw message