flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Re: Cannot instantiate Mysql connection
Date Thu, 25 Jun 2015 08:58:05 GMT
Sorry for the late response but I was on vacation the last 2 weeks..

Calling Class.forName("com.mysql.jdbc.Driver") in the main() of my class
made the things work!

Thanks for the support,
Flavio

On Fri, Jun 5, 2015 at 11:13 PM, Stephan Ewen <sewen@apache.org> wrote:

> Can you manually load the driver class, with "Class.forName(...)", or does
> that yield a "ClassNotFoundException" ?
>
> On Fri, Jun 5, 2015 at 11:10 PM, Flavio Pompermaier <pompermaier@okkam.it>
> wrote:
>
>> in the fat jar
>> On 5 Jun 2015 19:28, "Stephan Ewen" <sewen@apache.org> wrote:
>>
>>> In which way is the driver in the classpath?
>>>
>>>  - fat jar?
>>>  - in the nested /out folder in the slim jar?
>>>
>>>
>>>
>>> On Fri, Jun 5, 2015 at 7:23 PM, Flavio Pompermaier <pompermaier@okkam.it
>>> > wrote:
>>>
>>>> Actually I just need to load it in the main method (job manager) before
>>>> calling any flink operation, I retrieve the records in a mysql table
>>>> because they contain the path of files I'll need to read. Nothing more
>>>> nothing less
>>>> On 5 Jun 2015 19:06, "Robert Metzger" <rmetzger@apache.org> wrote:
>>>>
>>>>> Sure.
>>>>>
>>>>> So the DriverManager has a static variable called "registeredDrivers".
>>>>> When DriverManager.getConnection() is called, the method is looking
>>>>> up if an registered driver for that connection (in this case "mysql")
is
>>>>> available.
>>>>>
>>>>> For drivers to be in that list, they have to register themselves using
>>>>> the DriverManager.registerDriver() method.
>>>>>
>>>>> Drivers can register themselves with a static constructor (which is
>>>>> executed when Java loads a class):
>>>>>
>>>>> public class PoolingDriver implements Driver {
>>>>>     /** Register myself with the {@link DriverManager}. */
>>>>>     static {
>>>>>         try {
>>>>>             DriverManager.registerDriver(new PoolingDriver());
>>>>>         } catch(Exception e) {
>>>>>         }
>>>>>     }
>>>>>
>>>>> To execute that driver registration, you need to do:
>>>>> Class.forName("org.datanucleus.store.rdbms.datasource.dbcp.
>>>>> PoolingDriver");
>>>>> because then Java is loading the class and executing the static
>>>>> constructor which is registering the driver at the connection manager.
>>>>>
>>>>> When executing Flink locally, you are using only one JVM. By calling
>>>>> the MySQL driver manually in the main() method of your flink job, you
are
>>>>> registering the MySQL driver at the DriverManager of that JVM.
>>>>>
>>>>> However, when you run Flink in a distributed cluster, at the
>>>>> TaskManager JVMs, the MySQL driver is not loaded at the DriverManager
there.
>>>>> Therefore, you have to make sure that Class.forName("
>>>>> org.datanucleus.store.rdbms.datasource.dbcp.PoolingDriver"); (this is
>>>>> not the correct class for the MySQL driver)
>>>>> has been called.
>>>>> One approach to do that is to call Class.forName() in the open()
>>>>> method of your function.
>>>>>
>>>>> Best,
>>>>> Robert
>>>>>
>>>>>
>>>>> On Fri, Jun 5, 2015 at 6:54 PM, Flavio Pompermaier <
>>>>> pompermaier@okkam.it> wrote:
>>>>>
>>>>>> HI Robert,
>>>>>> In the main method I connect to a mysql table that acts as a
>>>>>> data-source repository that I use to know which dataset I need to
load. All
>>>>>> mysql classes are present in the shaded jar.
>>>>>> Could you explain a little bit more in detail the solution to fix
>>>>>> this problem please? Sorry but I didn't understand it :(
>>>>>>
>>>>>> Thanks,
>>>>>> Flavio
>>>>>> On 5 Jun 2015 18:33, "Robert Metzger" <rmetzger@apache.org>
wrote:
>>>>>>
>>>>>>> Hi Stefano,
>>>>>>>
>>>>>>> I doubt that there are conflicting dependencies because Flink
does
>>>>>>> not contain MySQL dependencies.
>>>>>>> Are you using Flink's JDBCInputFormat or custom code?
>>>>>>>
>>>>>>> For drivers to register at java.sql's DriverManager, their classes
>>>>>>> need to be loaded first. To load a class, you need to call
>>>>>>> Class.forName("<classname>");
>>>>>>>
>>>>>>> Maybe you are loading the class in the application's main() method
>>>>>>> (thats why it is working from eclipse) but not on the cluster
instances
>>>>>>> which are supposed to read the data.
>>>>>>>
>>>>>>> On Fri, Jun 5, 2015 at 5:16 PM, Stefano Bortoli <s.bortoli@gmail.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Hi Robert,
>>>>>>>>
>>>>>>>> I answer on behalf of Flavio. He told me the driver jar was
>>>>>>>> included. Smells lik class-loading issue due to 'conflicting'
>>>>>>>> dependencies.  Is it possible?
>>>>>>>>
>>>>>>>> Saluti,
>>>>>>>> Stefano
>>>>>>>>
>>>>>>>> 2015-06-05 16:24 GMT+02:00 Robert Metzger <rmetzger@apache.org>:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> is the MySQL driver part of the Jar file that you've
build?
>>>>>>>>>
>>>>>>>>> On Fri, Jun 5, 2015 at 4:11 PM, Flavio Pompermaier <
>>>>>>>>> pompermaier@okkam.it> wrote:
>>>>>>>>>
>>>>>>>>>> Hi to all,
>>>>>>>>>>
>>>>>>>>>> I'm using a fresh build of flink-0.9-SNAPSHOT and
in my flink job
>>>>>>>>>> I set up a mysql connection.
>>>>>>>>>> When I run the job from Eclipse everything is fine,
>>>>>>>>>> while when running the job from the Web UI I get
the following
>>>>>>>>>> exception:
>>>>>>>>>>
>>>>>>>>>> java.sql.SQLException: No suitable driver found for
>>>>>>>>>> jdbc:mysql:/localhost:3306/mydb?autoReconnect=true
>>>>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:596)
>>>>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:215)
>>>>>>>>>>
>>>>>>>>>> How can I fix that?
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Flavio
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>
>

Mime
View raw message