flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Foster, Craig" <foscr...@amazon.com>
Subject Re: flink-dist shading
Date Mon, 21 Nov 2016 17:27:08 GMT
Thanks for explaining, Robert and Gordon. For however it helps, I’ll comment on the original
Maven issue which seems to be affecting many people. I am trying to create RPMs using BigTop
and am still not quite sure how to handle this case. I guess what I could do is build the
parent first in our script and then build flink-dist again as described in the instructions.
I’ll try that out and see if it resolves the issue.

From: Robert Metzger <rmetzger@apache.org>
Reply-To: "user@flink.apache.org" <user@flink.apache.org>
Date: Saturday, November 19, 2016 at 4:08 AM
To: "user@flink.apache.org" <user@flink.apache.org>
Subject: Re: flink-dist shading

Hi Craig,
I also received only this email (and I'm a moderator of the dev@ list, so the message never
made it into Apache's infra)

When this issue was first reported [1][2] I asked on the Maven mailing list what's going on
[3]. I think this JIRA contains the most information on the issue: https://issues.apache.org/jira/browse/MNG-5899
It doesn't seem that Maven is going to fix the issue anytime soon.

One idea I had regarding this issue was to print a warning in the maven output if we detect
Maven 3.3 at build time.

[1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/flink-dist-packaging-including-unshaded-classes-td9504.html
[2] https://issues.apache.org/jira/browse/FLINK-3158
[3] http://comments.gmane.org/gmane.comp.jakarta.turbine.maven.user/137621

On Fri, Nov 18, 2016 at 7:23 PM, Tzu-Li (Gordon) Tai <tzulitai@apache.org<mailto:tzulitai@apache.org>>
Hi Craig,

I think the email wasn't sent to the ‘dev’ list, somehow.

Have you tried this:

mvn clean install -DskipTests

# In Maven 3.3 the shading of flink-dist doesn't work properly in one run, so we need to run
mvn for flink-dist again.

cd flink-dist

mvn clean install -DskipTests
I agree that it’ll affect downstream users who need to build Flink themselves, and would
be best if it can be resolved.
The above is still more or less a “workaround”, but since I don’t really know the reason
for why the newer Maven versions
won’t properly shade, we’ll probably need to wait for others more knowledgable on the
build infrastructure to chime in and
see if there’s a good long-term solution.

Best Regards,

On November 19, 2016 at 8:48:32 AM, Foster, Craig (foscraig@amazon.com<mailto:foscraig@amazon.com>)
I’m not even sure this was delivered to the ‘dev’ list but I’ll go ahead and forward
the same email to the ‘user’ list since I haven’t seen a response.

I’m following up on the issue in FLINK-5013 about flink-dist specifically requiring Maven
3.0.5 through to <3.3. This affects people who build Flink with BigTop (not only EMR),
so I’m wondering about the context and how we can properly shade the Apache HTTP libraries
so that flink-dist can be built with a current version of Maven. Any insight into this would
be helpful.


View raw message