flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Return of Flink shading problems in 1.2.0
Date Fri, 17 Mar 2017 11:08:23 GMT
Hi Craig!

Maven 3.3.x has a shading problem. You need to build two times, once from
root, once inside "flink-dist". Have a look here:

https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/building.html#dependency-shading

Maybe that way missed in BigTop?

I am wondering if we should actually throw an error if building with Maven
3.3.x - too many users run into that issue.

Stephan



On Fri, Mar 17, 2017 at 8:14 AM, Ufuk Celebi <uce@apache.org> wrote:

> Pulling in Robert and Stephan who know the project's shading setup the
> best.
>
> On Fri, Mar 17, 2017 at 6:52 AM, Foster, Craig <foscraig@amazon.com>
> wrote:
> > Hi:
> >
> > A few months ago, I was building Flink and ran into shading issues for
> > flink-dist as described in your docs. We resolved this in BigTop by
> adding
> > the correct way to build flink-dist in the do-component-build script and
> > everything was fine after that.
> >
> >
> >
> > Now, I’m running into issues doing the same now in Flink 1.2.0 and I’m
> > trying to figure out what’s changed and how to fix it. Here’s how the
> > flink-dist jar looks with proper shading:
> >
> >
> >
> > jar -tvf /usr/lib/flink/lib/flink-dist_2.10-1.1.4.jar | grep
> > HttpConnectionParams
> > 2485 Tue Jan 01 00:00:00 UTC 1980
> > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/
> HttpConnectionParams.class
> > 3479 Tue Jan 01 00:00:00 UTC 1980
> > org/apache/flink/hadoop/shaded/org/apache/http/params/
> HttpConnectionParams.class
> >
> >
> >
> > When I build Flink 1.2.0 in BigTop, here’s shading for the jar found in
> the
> > RPM:
> >
> >
> >
> > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams
> > 2392 Tue Jan 01 00:00:00 GMT 1980
> > org/apache/commons/httpclient/params/HttpConnectionParams.class
> > 2485 Tue Jan 01 00:00:00 GMT 1980
> > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/
> HttpConnectionParams.class
> > 3479 Tue Jan 01 00:00:00 GMT 1980
> > org/apache/flink/hadoop/shaded/org/apache/http/params/
> HttpConnectionParams.class
> > 2868 Tue Jan 01 00:00:00 GMT 1980
> > org/apache/http/params/HttpConnectionParams.class
> >
> >
> >
> > I thought maybe it was some strange thing going on with BigTop, so then I
> > tried just straight building Flink 1.2.0 (outside BigTop) and get the
> same
> > shading:
> >
> >
> >
> > jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams
> >
> >   2485 Fri Mar 17 05:41:16 GMT 2017
> > org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/
> HttpConnectionParams.class
> >
> >   3479 Fri Mar 17 05:41:16 GMT 2017
> > org/apache/flink/hadoop/shaded/org/apache/http/params/
> HttpConnectionParams.class
> >
> >   2392 Fri Mar 17 05:41:24 GMT 2017
> > org/apache/commons/httpclient/params/HttpConnectionParams.class
> >
> >   2868 Fri Mar 17 05:41:24 GMT 2017
> > org/apache/http/params/HttpConnectionParams.class
> >
> >
> >
> > And, yes, this is after going into flink-dist and running mvn clean
> install
> > again since I am using Maven 3.3.x.
> >
> >
> >
> > Here’s a snippet from my Maven version:
> >
> > mvn -version
> >
> > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
> > 2015-11-10T16:41:47+00:00)
> >
> > Maven home: /usr/local/apache-maven
> >
> > Java version: 1.8.0_121, vendor: Oracle Corporation
> >
> >
> >
> > Any ideas on what my problem might be here?
> >
> >
> >
> > Thanks,
> >
> > Craig
> >
> >
>

Mime
View raw message