flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Levin <aaronle...@stripe.com>
Subject Re: Does anyone have an example of Bazel working with Flink?
Date Thu, 18 Jun 2020 16:32:06 GMT
Hi Austin,

In our experience, `rules_scala` and `rules_java` are enough for us at this
point.

It's entirely possible I'm not thinking far enough into the future, though,
so don't take our lack of investment as a sign it's not worth investing in
:)

Best,

Aaron Levin

On Thu, Jun 18, 2020 at 10:27 AM Austin Cawley-Edwards <
austin.cawley@gmail.com> wrote:

> Great to hear Dan!
>
> @Aaron - would you/ your team be interested in a `rules_flink` project?
> I'm still fairly new to Bazel and know enough to contribute, but could
> definitely use guidance on design as well.
>
> Best,
> Austin
>
> On Mon, Jun 15, 2020 at 11:07 PM Dan Hill <quietgolfer@gmail.com> wrote:
>
>> Thanks for the replies!  I was able to use the provided answers to get a
>> setup working (maybe not the most efficiently).  The main change I made was
>> to switch to including the deploy jar in the image (rather than the default
>> one).
>>
>> I'm open to contributing to a "rules_flink" project.  I don't know enough
>> yet to help design it.
>>
>> On Sat, Jun 13, 2020 at 4:39 AM Till Rohrmann <trohrmann@apache.org>
>> wrote:
>>
>>> Hi Dan,
>>>
>>> if you want to run a Flink job without specifying the main class via
>>> `bin/flink run --class org.a.b.Foobar` then you have to add a MANIFEST.MF
>>> file to your jar under META-INF and this file needs to contain `Main-Class:
>>> org.a.b.Foobar`.
>>>
>>> Cheers,
>>> Till
>>>
>>> On Fri, Jun 12, 2020 at 12:30 AM Austin Cawley-Edwards <
>>> austin.cawley@gmail.com> wrote:
>>>
>>>> Hey all,
>>>>
>>>> Adding to Aaron's response, we use Bazel to build our Flink apps. We've
>>>> open-sourced some of our setup here[1] though a bit outdated. There are
>>>> definitely rough edges/ probably needs a good deal of work to fit other
>>>> setups. We have written a wrapper around the `java_library` and
>>>> `java_binary` and could do the same for `rules_scala`, though we just
>>>> started using Bazel last November and have a lot to learn in terms of best
>>>> practices there.
>>>>
>>>> If you're interested in contributing to a `rules_flink` project, I
>>>> would be as well!
>>>>
>>>> Best,
>>>> Austin
>>>>
>>>> [1]: https://github.com/fintechstudios/vp-operator-demo-ff-virtual-2020
>>>>
>>>> On Thu, Jun 11, 2020 at 6:14 PM Aaron Levin <aaronlevin@stripe.com>
>>>> wrote:
>>>>
>>>>> Hi Dan,
>>>>>
>>>>> We use Bazel to compile our Flink applications. We're using
>>>>> "rules_scala" (https://github.com/bazelbuild/rules_scala) to manage
>>>>> the dependencies and produce jars. We haven't had any issues. However,
I
>>>>> have found that sometimes it's difficult to figure out exactly what Flink
>>>>> target or dependency my application needs.
>>>>>
>>>>> Unfortunately I'm not sure what issue you're seeing here. I would
>>>>> guess either your flink application wasn't compiled into the jar
>>>>> you're executing. If you can paste the bazel target used to generate
your
>>>>> jar and how you're launching the application, that will be helpful
>>>>> for diagnosis.
>>>>>
>>>>> On Thu, Jun 11, 2020 at 5:21 PM Dan Hill <quietgolfer@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I took the Flink playground and I'm trying to swap out Maven for
>>>>>> Bazel.  I got to the point where I'm hitting the following error.
 I want
>>>>>> to diff my code with an existing, working setup.
>>>>>>
>>>>>> Thanks! - Dan
>>>>>>
>>>>>>
>>>>>> client_1                | org.apache.flink.client.program.ProgramInvocationException:
>>>>>> Neither a 'Main-Class', nor a 'program-class' entry was found in
the jar
>>>>>> file.
>>>>>>
>>>>>> client_1                | at
>>>>>> org.apache.flink.client.program.PackagedProgram.getEntryPointClassNameFromJar(PackagedProgram.java:596)
>>>>>>
>>>>>> client_1                | at
>>>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:190)
>>>>>>
>>>>>> client_1                | at
>>>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:128)
>>>>>>
>>>>>> client_1                | at
>>>>>> org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:862)
>>>>>>
>>>>>> client_1                | at
>>>>>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:204)
>>>>>>
>>>>>> client_1                | at
>>>>>> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
>>>>>>
>>>>>> client_1                | at
>>>>>> org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
>>>>>>
>>>>>> client_1                | at
>>>>>> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>>>>>>
>>>>>> client_1                | at
>>>>>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
>>>>>>
>>>>>

Mime
View raw message