flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxim Parkachov <lazy.gop...@gmail.com>
Subject Re: Ship compiled code with broadcast stream ?
Date Tue, 09 Oct 2018 13:08:06 GMT

This is certainly possible. What you can do is use a
> BroadcastProcessFunction where you receive the rule code on the broadcast
> side.

Yes, this part works, no problem.

> You probably cannot send newly compiled objects this way but what you can
> do is either send a reference to some compiled jars and load them with the
> URLClassloader or send the actual String code and invoke the java compiler
> from your function.

Compiling from source code ... actually, I can implement rules in nashorn
JavaScript, but I'm pretty sure I'll get not serializable exception, but
I'll check this.

Referencing existing jar will not solve problem as I would need to
re-submit job, which I want to avoid in the first place. I actually wanted
exactly first scenario, send newly compiled objects.



View raw message