flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Wang <hello123...@gmail.com>
Subject Re: flink run jar throw java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
Date Wed, 30 Oct 2019 09:55:23 GMT
Hello Yang, Frank
Thank you for your reply.

Frank I have created a fat jar called flink-sql-submit.jar , the file size
is 8.2M .

You can create a fat jar (also called Uber jar) that includes all
> dependencies in your application jar.
>
> I would avoid to put things in the Flink lib directory as it can make
> maintenance difficult. Eg deployment is challenging, upgrade of flink,
> providing it on new nodes etc.
>

I am in favor of your point of view, I want to maintain the project's
dependencies, rather than put it in the Flink lib directory.
I will continue to solve this problem. I will give you feedback if I have
an answer.

Thanks again, Yang and Frank


vino yang <yanghua1127@gmail.com> 于2019年10月30日周三 下午4:05写道:

> Hi Franke,
>
> From the information provided by Alex:
>
> >> mvn build jar include com.mysql.jdbc.Driver.
>
> it seems he has packaged a fat jar?
>
> Best,
> Vino
>
> Jörn Franke <jornfranke@gmail.com> 于2019年10月30日周三 下午2:47写道:
>
>>
>>
>> You can create a fat jar (also called Uber jar) that includes all
>> dependencies in your application jar.
>>
>> I would avoid to put things in the Flink lib directory as it can make
>> maintenance difficult. Eg deployment is challenging, upgrade of flink,
>> providing it on new nodes etc.
>>
>>
>> Am 30.10.2019 um 04:46 schrieb Alex Wang <hello123.cn@gmail.com>:
>>
>> 
>> Hello everyone, I am a newbie.
>> I am learning the flink-sql-submit project. From @Jark Wu  :
>> https://github.com/wuchong/flink-sql-submit
>>
>> My local environment is:
>> 1. flink1.9.0 standalone
>> 2. kafka_2.11-2.2.0 single
>>
>> I configured Flink Connectors and Formats jars to $FLINK_HOME/lib .
>> Reference:
>> https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/table/connect.html#connectors
>>
>> Then I run flink-sql-submit , sh run.sh q1
>> Throw  java.lang.ClassNotFoundException: com.mysql.jdbc.Driver.
>>
>> My question is:
>> I configured mysql-connector-java in the pom.xml file, mvn build jar
>> include com.mysql.jdbc.Driver.
>> Why is this error still reported? I put the jar package in
>> $FLINK_HOME/lib and the problem can be solved.
>> Do you need to put these jars in $FLINK_HOME/lib when the project relies
>> on too many jar packages?
>> If I don't put mysql-connector-java.jar in $FLINK_HOME/lib, how can I
>> solve this problem?
>>
>> Can @Jark Wu  give me some advice? Or can someone give me some advice?
>> Thank you.
>>
>> 1. pom.xml
>>
>> <dependency>
>>> <groupId>mysql</groupId>
>>> <artifactId>mysql-connector-java</artifactId>
>>> <version>5.1.38</version>
>>> </dependency>
>>
>> 2. mvn clean; mvn package
>>
>> $ ll -rth target
>>>
>>>  [±master ●]
>>> total 32312
>>> drwxr-xr-x  3 alex  staff    96B Oct 30 11:32 generated-sources
>>> drwxr-xr-x  5 alex  staff   160B Oct 30 11:32 classes
>>> drwxr-xr-x  3 alex  staff    96B Oct 30 11:32 maven-archiver
>>> -rw-r--r--  1 alex  staff   7.2M Oct 30 11:32
>>> flink-sql-submit-1.0-SNAPSHOT.jar
>>> -rw-r--r--  1 alex  staff   8.2M Oct 30 11:32 flink-sql-submit.jar
>>>
>>
>> 3. flink-sql-submit.jar include java.sql.Driver
>>
>> " zip.vim version v28
>>> " Browsing zipfile
>>> /Users/alex/IdeaProjects/alex/flink_learn/flink-sql-submit/target/flink-sql-submit.jar
>>> " Select a file with cursor and press ENTER
>>>
>>> META-INF/MANIFEST.MF
>>> META-INF/
>>> q1.sql
>>> user_behavior.log
>>> com/
>>> com/github/
>>> com/github/wuchong/
>>> com/github/wuchong/sqlsubmit/
>>> com/github/wuchong/sqlsubmit/SqlSubmit$1.class
>>> com/github/wuchong/sqlsubmit/SqlSubmit.class
>>> com/github/wuchong/sqlsubmit/SourceGenerator.class
>>> com/github/wuchong/sqlsubmit/cli/
>>> com/github/wuchong/sqlsubmit/cli/SqlCommandParser$SqlCommandCall.class
>>> com/github/wuchong/sqlsubmit/cli/SqlCommandParser.class
>>> com/github/wuchong/sqlsubmit/cli/SqlCommandParser$SqlCommand.class
>>> com/github/wuchong/sqlsubmit/cli/CliOptions.class
>>> com/github/wuchong/sqlsubmit/cli/CliOptionsParser.class
>>> META-INF/maven/
>>> META-INF/maven/com.github.wuchong/
>>> META-INF/maven/com.github.wuchong/flink-sql-submit/
>>> META-INF/maven/com.github.wuchong/flink-sql-submit/pom.xml
>>> META-INF/maven/com.github.wuchong/flink-sql-submit/pom.properties
>>> META-INF/services/
>>> META-INF/services/java.sql.Driver
>>> com/mysql/
>>> com/mysql/fabric/
>>> com/mysql/fabric/FabricCommunicationException.class
>>> com/mysql/fabric/FabricConnection.class
>>> com/mysql/fabric/FabricStateResponse.class
>>> com/mysql/fabric/HashShardMapping$ReverseShardIndexSorter.class
>>> com/mysql/fabric/HashShardMapping.class
>>> com/mysql/fabric/RangeShardMapping$RangeShardIndexSorter.class
>>> com/mysql/fabric/RangeShardMapping.class
>>> com/mysql/fabric/Response.class
>>> com/mysql/fabric/Server.class
>>> com/mysql/fabric/ServerGroup.class
>>> com/mysql/fabric/ServerMode.class
>>> com/mysql/fabric/ServerRole.class
>>> etc ...
>>>
>>
>>
>>
>> $FLINK_DIR/bin/flink run -d -p 3 target/flink-sql-submit.jar -w
>> "${PROJECT_DIR}"/src/main/resources/ -f "$1".sql
>> Eerror:
>> 2019-10-30 10:27:35
>> java.lang.IllegalArgumentException: JDBC driver class not found.
>> At
>> org.apache.flink.api.java.io.jdbc.JDBCUpsertOutputFormat.open(JDBCUpsertOutputFormat.java:112)
>> At
>> org.apache.flink.api.java.io.jdbc.JDBCUpsertSinkFunction.open(JDBCUpsertSinkFunction.java:42)
>> At
>> org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
>> At
>> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
>> At
>> org.apache.flink.streaming.api.operators.StreamSink.open(StreamSink.java:48)
>> At
>> org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:532)
>> At
>> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:396)
>> At org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
>> At org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
>> At java.lang.Thread.run(Thread.java:748)
>> Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
>> At java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>> At java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> At sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
>> At java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>> At java.lang.Class.forName0(Native Method)
>> At java.lang.Class.forName(Class.java:264)
>> At
>> org.apache.flink.api.java.io.jdbc.AbstractJDBCOutputFormat.establishConnection(AbstractJDBCOutputFormat.java:66)
>> At
>> org.apache.flink.api.java.io.jdbc.JDBCUpsertOutputFormat.open(JDBCUpsertOutputFormat.java:99)
>> ... 9 more
>>
>>
>> --
>> Best
>>
>>

-- 
Best

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message