flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Foster, Craig" <foscr...@amazon.com>
Subject Re: Return of Flink shading problems in 1.2.0
Date Fri, 17 Mar 2017 22:56:00 GMT
Ping. So I’ve built with 3.0.5 and it does give proper shading. So it does get me yet another
workaround where my only recourse is to use a max version of Maven. Still, I feel there should
be a long-term fix at some point in time.

I also believe there is a regression in Flink 1.2.0 for Maven 3.3.x with the process as documented,
so hoping someone can at least duplicate or let me know of a new workaround for 3.3.x.

Thanks!
Craig

From: "Foster, Craig" <foscraig@amazon.com>
Reply-To: "user@flink.apache.org" <user@flink.apache.org>
Date: Friday, March 17, 2017 at 7:23 AM
To: "user@flink.apache.org" <user@flink.apache.org>
Cc: Ufuk Celebi <uce@apache.org>, Robert Metzger <rmetzger@apache.org>, Stephan
Ewen <sewen@apache.org>
Subject: Re: Return of Flink shading problems in 1.2.0

Hey Stephen:
I am building twice in every case described in my previous mail. Well, building then rebuilding
the flink-dist submodule.

This was fixed in BigTop but I started seeing this issue again with Flink 1.2.0. I was wondering
if there's something else in the environment that could prevent the shading from working because
it isn't now even with the workaround.

On Mar 17, 2017, at 4:08 AM, Stephan Ewen <sewen@apache.org<mailto:sewen@apache.org>>
wrote:
Hi Craig!

Maven 3.3.x has a shading problem. You need to build two times, once from root, once inside
"flink-dist". Have a look here:

https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/building.html#dependency-shading

Maybe that way missed in BigTop?

I am wondering if we should actually throw an error if building with Maven 3.3.x - too many
users run into that issue.

Stephan



On Fri, Mar 17, 2017 at 8:14 AM, Ufuk Celebi <uce@apache.org<mailto:uce@apache.org>>
wrote:
Pulling in Robert and Stephan who know the project's shading setup the best.

On Fri, Mar 17, 2017 at 6:52 AM, Foster, Craig <foscraig@amazon.com<mailto:foscraig@amazon.com>>
wrote:
> Hi:
>
> A few months ago, I was building Flink and ran into shading issues for
> flink-dist as described in your docs. We resolved this in BigTop by adding
> the correct way to build flink-dist in the do-component-build script and
> everything was fine after that.
>
>
>
> Now, I’m running into issues doing the same now in Flink 1.2.0 and I’m
> trying to figure out what’s changed and how to fix it. Here’s how the
> flink-dist jar looks with proper shading:
>
>
>
> jar -tvf /usr/lib/flink/lib/flink-dist_2.10-1.1.4.jar | grep
> HttpConnectionParams
> 2485 Tue Jan 01 00:00:00 UTC 1980
> org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class
> 3479 Tue Jan 01 00:00:00 UTC 1980
> org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class
>
>
>
> When I build Flink 1.2.0 in BigTop, here’s shading for the jar found in the
> RPM:
>
>
>
> jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams
> 2392 Tue Jan 01 00:00:00 GMT 1980
> org/apache/commons/httpclient/params/HttpConnectionParams.class
> 2485 Tue Jan 01 00:00:00 GMT 1980
> org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class
> 3479 Tue Jan 01 00:00:00 GMT 1980
> org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class
> 2868 Tue Jan 01 00:00:00 GMT 1980
> org/apache/http/params/HttpConnectionParams.class
>
>
>
> I thought maybe it was some strange thing going on with BigTop, so then I
> tried just straight building Flink 1.2.0 (outside BigTop) and get the same
> shading:
>
>
>
> jar -tvf flink-dist_2.10-1.2.0.jar | grep HttpConnectionParams
>
>   2485 Fri Mar 17 05:41:16 GMT 2017
> org/apache/flink/hadoop/shaded/org/apache/commons/httpclient/params/HttpConnectionParams.class
>
>   3479 Fri Mar 17 05:41:16 GMT 2017
> org/apache/flink/hadoop/shaded/org/apache/http/params/HttpConnectionParams.class
>
>   2392 Fri Mar 17 05:41:24 GMT 2017
> org/apache/commons/httpclient/params/HttpConnectionParams.class
>
>   2868 Fri Mar 17 05:41:24 GMT 2017
> org/apache/http/params/HttpConnectionParams.class
>
>
>
> And, yes, this is after going into flink-dist and running mvn clean install
> again since I am using Maven 3.3.x.
>
>
>
> Here’s a snippet from my Maven version:
>
> mvn -version
>
> Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
> 2015-11-10T16:41:47+00:00)
>
> Maven home: /usr/local/apache-maven
>
> Java version: 1.8.0_121, vendor: Oracle Corporation
>
>
>
> Any ideas on what my problem might be here?
>
>
>
> Thanks,
>
> Craig
>
>

Mime
View raw message