flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: Testing Apache Flink 0.9.0-rc1
Date Fri, 12 Jun 2015 08:42:46 GMT
I think I found a real release blocker. Currently we don't add license
files to our shaded jars. For example
the flink-shaded-include-yarn-0.9.0-milestone-1.jar shades hadoop code.
This code also includes the `org.apache.util.bloom.*` classes. These
classes are licensed under  The European Commission project OneLab. We have
a notice in the LICENSE file of our binary distribution but I think we also
have to add them in the shaded jar. There might even be more code bundled
as part of some shaded jars which I have not spotted yet.

Furthermore, I noticed that we list all Apache License dependencies in our
LICENSE file of our binary distribution (which we don't have to do).
However, we don't do it in our jars which contain for example guava and asm
as shaded dependencies. Maybe we should be consistent here.

But maybe I overlook something here and we don't have to do it.

On Fri, Jun 12, 2015 at 10:29 AM Till Rohrmann <trohrmann@apache.org> wrote:

> Well I think the initial idea was to keep the dist jar as small a possible
> and therefore we did not include the libraries. I'm not sure whether we can
> decide this here ad-hoc. If the community says that we shall include these
> libraries then I can add them. But bear in mind that all of them have some
> transitive dependencies which will be added as well.
>
>
> On Fri, Jun 12, 2015 at 10:15 AM Márton Balassi <balassi.marton@gmail.com>
> wrote:
>
>> As for outstanding issues I think streaming is good to go as far as I
>> know.
>> I am personally against including all libraries - at least speaking for
>> the
>> streaming connectors. Robert, Stephan and myself had a detailed discussion
>> on that some time ago and the disadvantage of having all the libraries in
>> the distribution is the dependency mess that they pull. In this case I
>> would rather add documentation on putting them in the user jar then. As
>> for
>> the other libraries they do not depend on so much external code, so +1 for
>> putting them in.
>>
>> On Fri, Jun 12, 2015 at 10:02 AM, Maximilian Michels <mxm@apache.org>
>> wrote:
>>
>> > We should have a nightly cluster test for every library. Let's keep
>> that in
>> > mind for the future. Very nice find, Till!
>> >
>> > Since there were not objections, I cherry-picked the proposed commits
>> from
>> > the document to the release-0.9 branch. If I understand correctly, we
>> can
>> > create the new release candidate once Till has checked the licenses,
>> Ufuk's
>> > TableInput fix has been merged, and Fabian's web interface improvement
>> are
>> > in. Plus, we need to include all Flink libraries in flink-dist. Are you
>> > going to fix that as well, Till?
>> >
>> > On Fri, Jun 12, 2015 at 9:53 AM, Ufuk Celebi <uce@apache.org> wrote:
>> >
>> > >
>> > > On 12 Jun 2015, at 09:45, Till Rohrmann <trohrmann@apache.org> wrote:
>> > >
>> > > > Hi guys,
>> > > >
>> > > > I just noticed while testing the TableAPI on the cluster that it is
>> not
>> > > > part of the dist module. Therefore, programs using the TableAPI will
>> > only
>> > > > run when you put the TableAPI jar directly on the cluster or if you
>> > > build a
>> > > > fat jar including the TableAPI jar. This is nowhere documented.
>> > > > Furthermore, this also applies to Gelly and FlinkML.
>> > >
>> > > I think all of these should be included in the fat jar. They are all
>> > > highly advertized components.
>> > >
>> > > Very good catch, Till! I didn't get around to testing Table API on a
>> > > cluster, yet.
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message