Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CANZa=GvG71JFwuNsbtH9JH4-fR9EGONd1GYwaoPNmY8d6SFQ3g@mail.gmail.com>
References: <5669A2AD.5010600@apache.org>
 <0EF1168A-78D8-499E-A45A-811A0E599DC8@apache.org>
 <5669A509.3080509@apache.org>
 <CANC1h_tGsvV3VUb-rGiK3T0umTbFnq-Eqg+w0XohkrHosyeahg@mail.gmail.com>
 <CANZa=GvG71JFwuNsbtH9JH4-fR9EGONd1GYwaoPNmY8d6SFQ3g@mail.gmail.com>
From: Robert Metzger <rmetzger@apache.org>
Date: Thu, 10 Dec 2015 17:54:51 +0100
Message-ID: 
 <CAGr9p8BkfttW42i6wN1nKUjx3ax9F38JROB-CjF2v-9vJeXq9A@mail.gmail.com>
Subject: Re: Flink and Clojure
To: "dev@flink.apache.org" <dev@flink.apache.org>
Content-Type: multipart/alternative; boundary=001a11401752dff6ac05268e116b

--001a11401752dff6ac05268e116b
Content-Type: text/plain; charset=UTF-8

I had the same though as Nick. Maybe Leiningen allows to somehow build a
fat-jar containing the clojure standard library.

On Thu, Dec 10, 2015 at 5:51 PM, Nick Dimiduk <ndimiduk@apache.org> wrote:

> What happens when you follow the packaging examples provided in the flink
> quick start archetypes? These have the maven-foo required to package an
> uberjar suitable for flink submission. Can you try adding that step to your
> pom.xml?
>
> On Thursday, December 10, 2015, Stephan Ewen <sewen@apache.org> wrote:
>
> > This is a problem in Java.
> > I think you cannot dynamically modify the initial system class loader.
> >
> > What most apps do is check for the thread context class loader when
> > dynamically loading classes. We can check and make sure that one is set,
> > but if Closure does not respect that, we have a problem.
> > Then Closure is not built for dynamic class loading.
> >
> >
> >
> > On Thu, Dec 10, 2015 at 5:15 PM, Matthias J. Sax <mjsax@apache.org
> > <javascript:;>> wrote:
> >
> > > Would it make sense (if possible?) for Flink to add the user jar
> > > dynamically to it's own classpath so Clojure can find it? Or somehow
> > > modify Clojure's class loader?
> > >
> > > The jars in lib are added to the classpath at startup. This makes it
> > > practically impossible to execute a Flink program that is written in
> > > Clojure right now...
> > >
> > >
> > > On 12/10/2015 05:09 PM, Aljoscha Krettek wrote:
> > > > Clojure is not considering the user-jar when trying to load the
> class.
> > > >
> > > >> On 10 Dec 2015, at 17:05, Matthias J. Sax <mjsax@apache.org
> > <javascript:;>> wrote:
> > > >>
> > > >> Hi Squirrels,
> > > >>
> > > >> I was playing with a Flink Clojure WordCount example today.
> > > >> https://github.com/mjsax/flink-external/tree/master/flink-clojure
> > > >>
> > > >> After building the project with "mvn package" I tried to submit it
> to
> > a
> > > >> local cluster. Before I started the cluster, I manually copied
> > > >> "clojure-1.5.1.jar" into Flink's lib folder.
> > > >>
> > > >>> cp ~/.m2/repository/org/clojure/clojure/1.5.1/clojure-1.5.1.jar
> lib/
> > > >>> bin/start-local.sh
> > > >>
> > > >> However, when submitting the jar, I get an exception:
> > > >>
> > > >>> bin/flink run -c org.apache.flink.clojure.WordCount
> > > >>
> > >
> >
> ~/workspace_flink/flink-external/flink-clojure/target/flink-clojure-0.10.0.jar
> > > >>
> > > >>
> > > >>> ------------------------------------------------------------
> > > >>> The program finished with the following exception:
> > > >>>
> > > >>> org.apache.flink.client.program.ProgramInvocationException: The
> > > program's entry point class 'org.apache.flink.clojure.WordCount' threw
> an
> > > error during initialization.
> > > >>> at
> > >
> >
> org.apache.flink.client.program.PackagedProgram.loadMainClass(PackagedProgram.java:585)
> > > >>> at
> > >
> >
> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:195)
> > > >>> at
> > > org.apache.flink.client.CliFrontend.buildProgram(CliFrontend.java:784)
> > > >>> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:288)
> > > >>> at
> > >
> >
> org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1050)
> > > >>> at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1103)
> > > >>> Caused by: java.lang.ExceptionInInitializerError
> > > >>> at java.lang.Class.forName0(Native Method)
> > > >>> at java.lang.Class.forName(Class.java:278)
> > > >>> at
> > >
> >
> org.apache.flink.client.program.PackagedProgram.loadMainClass(PackagedProgram.java:578)
> > > >>> ... 5 more
> > > >>> Caused by: java.io.FileNotFoundException: Could not locate
> > > org/apache/flink/clojure/WordCount__init.class or
> > > org/apache/flink/clojure/WordCount.clj on classpath:
> > > >>> at clojure.lang.RT.load(RT.java:443)
> > > >>> at clojure.lang.RT.load(RT.java:411)
> > > >>> at clojure.core$load$fn__5018.invoke(core.clj:5530)
> > > >>> at clojure.core$load.doInvoke(core.clj:5529)
> > > >>> at clojure.lang.RestFn.invoke(RestFn.java:408)
> > > >>> at clojure.lang.Var.invoke(Var.java:415)
> > > >>> at org.apache.flink.clojure.WordCount.<clinit>(Unknown Source)
> > > >>> ... 8 more
> > > >>
> > > >> I am not sure why the class is not found. It is contained in the jar
> > > >> file. I can fix this error by copying the user jar
> > > >> (flink-clojure-0.10.0.jar) into Flink's lib-folder.
> > > >>
> > > >> It seems, that Flink is not looking into the user-jar when loading
> > this
> > > >> class. Can anybody explain why?
> > > >>
> > > >> Thx.
> > > >>
> > > >> -Matthias
> > > >>
> > > >
> > >
> > >
> >
>

--001a11401752dff6ac05268e116b--