flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Giacomo Licari <giacomo.lic...@gmail.com>
Subject Re: Compile flink file with external jar
Date Mon, 21 Sep 2015 17:49:20 GMT
Thanks a lot Fabian,
I will try it.

Cheers,
Giacomo

On Mon, Sep 21, 2015 at 7:35 PM, Fabian Hueske <fhueske@gmail.com> wrote:

> Hi Giacomo,
>
> you could use Janino [1] to directly compile the code string into a class
> and execute it. The program does not need to be shipped to the cluster if
> all user functions are contained in the jar.
>
> Cheers, Fabian
>
> [1] http://unkrig.de/w/Janino
> On Sep 21, 2015 7:08 PM, "Giacomo Licari" <giacomo.licari@gmail.com>
> wrote:
>
>> Hi Robert,
>> thanks for the reply.
>> I receive a JSON from my client interface, which contains the dataflow
>> description.
>> Then I parse that JSON and the parser creates a string which contains the
>> Flink code, as the user can modify the dataflow, the description can change
>> every time it calls "Execute Dataflow".
>>
>> Now I should be able to execute the generated String code, so I was first
>> thinking about to write it to a .java file.
>> What could you suggest to me? Is it possible to execute the string code
>> directly from the backend.jar I have?
>>
>>
>> On Mon, Sep 21, 2015 at 6:54 PM, Robert Metzger <rmetzger@apache.org>
>> wrote:
>>
>>> Hi,
>>>
>>> you have to make sure that the Flink classes are contained in your class
>>> path.
>>> Either add the flink-dist jar from the binary distribution to your class
>>> path, or use maven to build the backend.jar as a fat jar.
>>>
>>> Why are you generating a java class from your dataflows?
>>> Isn't it easier to just call the Flink APIs when parsing your flow
>>> description language?
>>>
>>> Robert
>>>
>>>
>>> On Mon, Sep 21, 2015 at 6:41 PM, Giacomo Licari <
>>> giacomo.licari@gmail.com> wrote:
>>>
>>>> Hi guys,
>>>>
>>>> I'm developing a dataflow client whose backend, exported as jar, allows
>>>> users to convert dataflows to a Flink file .java with some code inside. The
>>>> generated file naturally calls some classes I have in jar file, like
>>>> MapFunction, DataSet.
>>>>
>>>> My question is:
>>>> How to compile the generated file and pack it into a jar file, so I can
>>>> execute it with Flink?
>>>>
>>>> I have tried several methods like:
>>>>
>>>>    - javac -cp backend.jar:. GeneratedClass.java
>>>>    - javac -cp ".:backend.jar" GeneratedClass.java
>>>>    - javac -cp ".:./backend.jar" GeneratedClass.java
>>>>
>>>> with no success.
>>>>
>>>> I receive errors like:
>>>> GeneratedClass.java:2: error: package
>>>> org.apache.flink.api.common.functions does not exist GeneratedClass.java:3:
>>>> error: package org.apache.flink.api.java does not exist
>>>>
>>>>
>>>> Thanks a lot,
>>>>
>>>> Giacomo
>>>>
>>>
>>>
>>

Mime
View raw message