flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Richter <s.rich...@data-artisans.com>
Subject Re: [BUG?] Cannot Load User Class on Local Environment
Date Tue, 25 Apr 2017 10:00:19 GMT
Hi,

I would expect that the local environment picks up the class path from the code that launched
it. So I think the question is what happens behind the scenes when you call ignite.compute().broadcast(runnable);
. Which classes are shipped and how is the classpath build in the environment that runs the
code. Your example is also not fully conclusive, because com.myproj.Test (which you can successfully
instantiate) and com.myproj.Test$1$2 (which fails) are different classes, so maybe only the
outer class is shipped with the broadcast call. My theory is that not all classes are shipped
(e.g. inner classes), but only Test . You could try three things to analyze to problem a little
more:

1) Create another inner class inside Test and try if you are still able to instantiate also
this class via reflection.
2) Let Test class itself implement the map function (avoiding the usage of other/inner classes)
and see if this works.
3) Check and set the thread’s context classloader inside the runnable to something that
contains all required classes and see if this gets picked up by Flink.

Best,
Stefan

> Am 25.04.2017 um 07:27 schrieb Matt <dromitlabs@gmail.com>:
> 
> Hi all,
> 
> I'm trying to run Flink using a local environment, but on an Ignite node to achieve collocation
(as mentioned in my previous message on this list).
> 
> Have a look at the code in [1]. It's pretty simple, but I'm getting a "cannot load user
class" error as shown in [2].
> 
> If you check line #29 on the code, I'm able to create an instance of class Test, and
it's the same context from which I'm creating the Flink job. Shouldn't it work provided I'm
using a local environment?
> 
> It would be really nice to be able to inject a ClassLoader into the chunk of code that
creates the job. Is this currently possible?
> 
> Any fix or workaround is appreciated!
> 
> Best,
> Matt
> 
> [1] https://gist.github.com/f248187b9638023b95ba8bd9d7f06215 <https://gist.github.com/f248187b9638023b95ba8bd9d7f06215>
> [2] https://gist.github.com/796ee05425535ece1736df7b1e884cce <https://gist.github.com/796ee05425535ece1736df7b1e884cce>

Mime
View raw message