flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: Compile flink file with external jar
Date Mon, 21 Sep 2015 17:35:49 GMT
Hi Giacomo,

you could use Janino [1] to directly compile the code string into a class
and execute it. The program does not need to be shipped to the cluster if
all user functions are contained in the jar.

Cheers, Fabian

[1] http://unkrig.de/w/Janino
On Sep 21, 2015 7:08 PM, "Giacomo Licari" <giacomo.licari@gmail.com> wrote:

> Hi Robert,
> thanks for the reply.
> I receive a JSON from my client interface, which contains the dataflow
> description.
> Then I parse that JSON and the parser creates a string which contains the
> Flink code, as the user can modify the dataflow, the description can change
> every time it calls "Execute Dataflow".
>
> Now I should be able to execute the generated String code, so I was first
> thinking about to write it to a .java file.
> What could you suggest to me? Is it possible to execute the string code
> directly from the backend.jar I have?
>
>
> On Mon, Sep 21, 2015 at 6:54 PM, Robert Metzger <rmetzger@apache.org>
> wrote:
>
>> Hi,
>>
>> you have to make sure that the Flink classes are contained in your class
>> path.
>> Either add the flink-dist jar from the binary distribution to your class
>> path, or use maven to build the backend.jar as a fat jar.
>>
>> Why are you generating a java class from your dataflows?
>> Isn't it easier to just call the Flink APIs when parsing your flow
>> description language?
>>
>> Robert
>>
>>
>> On Mon, Sep 21, 2015 at 6:41 PM, Giacomo Licari <giacomo.licari@gmail.com
>> > wrote:
>>
>>> Hi guys,
>>>
>>> I'm developing a dataflow client whose backend, exported as jar, allows
>>> users to convert dataflows to a Flink file .java with some code inside. The
>>> generated file naturally calls some classes I have in jar file, like
>>> MapFunction, DataSet.
>>>
>>> My question is:
>>> How to compile the generated file and pack it into a jar file, so I can
>>> execute it with Flink?
>>>
>>> I have tried several methods like:
>>>
>>>    - javac -cp backend.jar:. GeneratedClass.java
>>>    - javac -cp ".:backend.jar" GeneratedClass.java
>>>    - javac -cp ".:./backend.jar" GeneratedClass.java
>>>
>>> with no success.
>>>
>>> I receive errors like:
>>> GeneratedClass.java:2: error: package
>>> org.apache.flink.api.common.functions does not exist GeneratedClass.java:3:
>>> error: package org.apache.flink.api.java does not exist
>>>
>>>
>>> Thanks a lot,
>>>
>>> Giacomo
>>>
>>
>>
>

Mime
View raw message