Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <1456161628001-5078.post@n4.nabble.com>
References: <1455313448712-4897.post@n4.nabble.com>
	<CANC1h_u5wCd8LkwFpb+-6B6AkMHG2sQr5RzF4o3uscVTUe+k=A@mail.gmail.com>
	<1455547299823-4924.post@n4.nabble.com>
	<CANC1h_uAsmq1NZCU87Z+QE-wKusKg_Je7nMoPz89hWbPi=+-LA@mail.gmail.com>
	<CAC27z=Pf6WdzQztS_VK0SExK_fv6pao_j3P38ZJc5eAM=CCAOw@mail.gmail.com>
	<1456161628001-5078.post@n4.nabble.com>
Date: Mon, 22 Feb 2016 19:01:53 +0100
Message-ID: 
 <CAC27z=OO+Ya9G5LigREnA5mAgwNzOuh0jrqb_bGF2LZOsB5_vg@mail.gmail.com>
Subject: Re: Flink packaging makes life hard for SBT fat jar's
From: Till Rohrmann <trohrmann@apache.org>
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=001a114af92ab7c4e4052c5fa09b

--001a114af92ab7c4e4052c5fa09b
Content-Type: text/plain; charset=UTF-8

Hi Shikhar,

you're right that including a connector dependency would have let us spot
the problem earlier. In fact, any project building a fat jar with SBT would
have failed without setting the flink dependencies to provided.

The problem is that the template is a general purpose template. Thus, it is
also used for batch jobs. I fear that by including a connector in the
default `build.sbt` file, many users will forget about it and simply
include it in their job jars. I admit that I'm not totally consistent with
my argumentation here because we're also including the `flink-scala` and
`flink-streaming-scala` dependency per default. But I would like to keep
the initial list of dependencies as lean as possible.

What I've done, however, is to add testing SBT builds with a connector to
our release testing tasks. This should catch similar problems which you've
came across.

Cheers,
Till

On Mon, Feb 22, 2016 at 6:20 PM, shikhar <shikhar@schmizz.net> wrote:

> Hi Till,
>
> Thanks so much for sorting this out!
>
> One suggestion, can the Flink template depend on a connector (Kafka?) --
> this would verify that assembly works smoothly for a very common use-case
> when you need to include connector JAR's.
>
> Cheers,
>
> Shikhar
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-packaging-makes-life-hard-for-SBT-fat-jar-s-tp4897p5078.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>

--001a114af92ab7c4e4052c5fa09b
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Shikhar,<div><br></div><div>you&#39;re right that inclu=
ding a connector dependency would have let us spot the problem earlier. In =
fact, any project building a fat jar with SBT would have failed without set=
ting the flink dependencies to provided.</div><div><br></div><div>The probl=
em is that the template is a general purpose template. Thus, it is also use=
d for batch jobs. I fear that by including a connector in the default `buil=
d.sbt` file, many users will forget about it and simply include it in their=
 job jars. I admit that I&#39;m not totally consistent with my argumentatio=
n here because we&#39;re also including the `flink-scala` and `flink-stream=
ing-scala` dependency per default. But I would like to keep the initial lis=
t of dependencies as lean as possible.</div><div><br></div><div>What I&#39;=
ve done, however, is to add testing SBT builds with a connector to our rele=
ase testing tasks. This should catch similar problems which you&#39;ve came=
 across.</div><div><br></div><div>Cheers,</div><div>Till</div></div><div cl=
ass=3D"gmail_extra"><br><div class=3D"gmail_quote">On Mon, Feb 22, 2016 at =
6:20 PM, shikhar <span dir=3D"ltr">&lt;<a href=3D"mailto:shikhar@schmizz.ne=
t" target=3D"_blank">shikhar@schmizz.net</a>&gt;</span> wrote:<br><blockquo=
te class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc so=
lid;padding-left:1ex">Hi Till,<br>
<br>
Thanks so much for sorting this out!<br>
<br>
One suggestion, can the Flink template depend on a connector (Kafka?) --<br=
>
this would verify that assembly works smoothly for a very common use-case<b=
r>
when you need to include connector JAR&#39;s.<br>
<br>
Cheers,<br>
<br>
Shikhar<br>
<br>
<br>
<br>
--<br>
View this message in context: <a href=3D"http://apache-flink-user-mailing-l=
ist-archive.2336050.n4.nabble.com/Flink-packaging-makes-life-hard-for-SBT-f=
at-jar-s-tp4897p5078.html" rel=3D"noreferrer" target=3D"_blank">http://apac=
he-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-packaging-ma=
kes-life-hard-for-SBT-fat-jar-s-tp4897p5078.html</a><br>
<div class=3D"HOEnZb"><div class=3D"h5">Sent from the Apache Flink User Mai=
ling List archive. mailing list archive at Nabble.com.<br>
</div></div></blockquote></div><br></div>

--001a114af92ab7c4e4052c5fa09b--