flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Kidder <kidder.sc...@gmail.com>
Subject Re: Proper way of adding external jars
Date Mon, 14 Nov 2016 17:49:26 GMT
Hi Gyula,

I've typically added external library dependencies to my own application
JAR as shaded-dependencies. This ensures that all dependencies are included
with my application while being distributed to Flink Job Manager & Task
Manager instances.

Another approach is to place these external JARs in the 'lib' sub-directory
of your Flink installation. Keep in mind that the external JARs must be
installed on every Flink node where your application is expected to run.
This works well for dependencies that are large in size or used by multiple
Flink applications in your cluster (avoid duplication of dependencies).

Best,

--Scott Kidder

On Mon, Nov 14, 2016 at 7:59 AM, Gyula Fóra <gyula.fora@gmail.com> wrote:

> Hi,
>
> I have been trying to use the -C flag to add external jars with user code
> and I have observed some strange behaviour.
>
> What I am trying to do is the following:
> I have 2 jars, JarWithMain.jar contains the main class and UserJar.jar
> contains some classes that the main method will eventually execute and also
> depends on classes from JarWithMain.
>
> Running this works:
> flink run .... -C UserJar.jar -c MainMethod JarWithMain.jar args...
>
> Running this leads to no class def found errors in the StreamTask
> initialization where it reads the functions from the config:
> flink run .... -C JarWithMain.jar -c MainMethod UserJar.jar  args...
>
> Did I miss something?
>
> Cheers,
> Gyula
>

Mime
View raw message