maven-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mmccarthy@tribloom.com" <mmccar...@tribloom.com>
Subject Re: Maven Shad Plugin with AWS Java SDK
Date Sat, 27 Jun 2015 17:22:26 GMT
Curtis,

I have checked in my code to github: 
git@github.com:tribloom/aws-sdk-java.git.

You can check out my shaded version by doing a "git clone 
git@github.com:tribloom/aws-sdk-java.git" followed by a "git checkout -b 
test origin/1.10.2" ("test" can be changed to whatever branch name you 
would like to use locally) in the newly created repository directory 
(aws-java-sdk).

You should see three modified files, pom.xml, 
./aws-java-sdk-core/pom.xml, and ./aws-java-sdk-s3/pom.xml.

Then run maven to build the project (narrowed down to just 
aws-java-sdk-core, aws-java-sdk-kms, and aws-java-sdk-s3 using the top 
level pom.xml):
mvn install -DskipTests=true

You should get the core and kms to build properly and then s3 will fail 
with:
[INFO] AWS SDK for Java .................................. SUCCESS [0.380s]
[INFO] AWS SDK for Java - Core ........................... SUCCESS [10.438s]
[INFO] AWS Java SDK for AWS KMS .......................... SUCCESS [2.942s]
[INFO] AWS Java SDK for Amazon S3 ........................ FAILURE [3.733s]
[INFO] 
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] 
------------------------------------------------------------------------
[INFO] Total time: 17.646s
[INFO] Finished at: Sat Jun 27 11:12:55 MDT 2015
[INFO] Final Memory: 33M/518M
[INFO] 
------------------------------------------------------------------------
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:2.3:compile 
(default-compile) on project aws-java-sdk-s3: Compilation failure
[ERROR] 
\dev\CSJ\aws\temp\aws-sdk-java\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26]

error: no suitable constructor found for 
S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase)

The way I understood the shade plugin to work, it would relocate the 
dependencies and "rewrite" the code that called those dependencies. I 
thought it would "fix" the constructor for S3ObjectInputStream to match 
the newly relocated org.apache.http.client.methods.HttoRequestBase 
class. If this isn't the way it should work, how then do I get the 
shaded project to build?

Thanks,
Michael
On 6/26/2015 5:52 AM, Curtis Rueden wrote:
> Hi Michael,
>
> Your error message refers to the maven-compiler-plugin. Are you sure the
> shade plugin is the culprit? Normally shade is bound to a later phase. If
> you need further detailed assistance, share your project (or an MCVE) with
> us.
>
> Regards,
> Curtis
> On Jun 25, 2015 5:16 PM, "mmccarthy@tribloom.com" <mmccarthy@tribloom.com>
> wrote:
>
>>   Matt/Curtis,
>>
>> I tried using both packages and a higher package with no luck, I get the
>> same error message:
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-compiler-plugin:2.3:compile
>> (default-compile) on project aws-java-sdk-s3: Compilation failure[ERROR]
>> \dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26]
>> error: no suitable constructor found for
>> S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase)
>>
>> I tried:
>>                      <relocations>
>>                          <relocation>
>>                              <pattern>org</pattern>
>>                              <shadedPattern>shaded</shadedPattern>
>>                          </relocation>
>>                      </relocations>
>> and:
>>                      <relocations>
>>                          <relocation>
>>                              <pattern>org.apache</pattern>
>>                              <shadedPattern>org.shaded</shadedPattern>
>>                          </relocation>
>>                          <relocation>
>>                              <pattern>org.apache.http</pattern>
>>                              <shadedPattern>org.shaded.http</shadedPattern>
>>                          </relocation>
>>                          <relocation>
>>                              <pattern>org.apache.http.methods</pattern>
>>
>> <shadedPattern>org.shaded.http.methods</shadedPattern>
>>                          </relocation>
>>                      </relocations>
>>
>> Has anyone else had luck with trying to shade the AWS Java SDK? Can anyone
>> with more experience try and build the project given my configuration and
>> tell me the results?
>>
>> Thanks,
>> Michael
>>   On 6/25/2015 10:27 AM, Curtis Rueden wrote:
>>
>> Hi Matt and Michael,
>>
>>   Matt Benson wrote:
>>> In my experience you must relocate each individual package; simply
>>> relocating org.apache.http is not sufficient.
>>   In my experience, the maven-shade-plugin will process subpackages which
>> match the given patterns. Here is a working example that shades all of
>> Jython's dependencies:
>>
>>
>> https://github.com/scijava/jython-shaded/blob/jython-shaded-2.5.3/deps/pom.xml#L56-L93
>>
>>   Regards,
>> Curtis
>>
>> On Thu, Jun 25, 2015 at 7:25 AM, Matt Benson <gudnabrsam@gmail.com> wrote:
>>
>>> In my experience you must relocate each individual package; simply
>>> relocating org.apache.http is not sufficient.
>>>
>>> Matt
>>> On Jun 25, 2015 6:11 AM, "mmccarthy@tribloom.com" <mmccarthy@tribloom.com
>>> wrote:
>>>
>>>> I have been attempting to "shade"[1] the AWS Java SDK[2] version 1.10.1,
>>>> specifically the S3 SDK, to relocate the Apache HTTPCompontents[3]
>>>> libraries. I am attempting to do this in order to resolve a dependency
>>>> conflict with another more complicated project which is also dependent
>>> on
>>>> an older version of HTTPComponents. So far I have been unsuccessful and
>>>> can't determine if it is misconfiguration or an issue with the plugin. I
>>>> hope it is the former.
>>>>
>>>> First I added a Maven Shade configuration to the aws-java-sdk-core
>>> pom.xml:
>>>> <plugin>
>>>>     <groupId>org.apache.maven.plugins</groupId>
>>>>     <artifactId>maven-shade-plugin</artifactId>
>>>>     <version>2.4</version>
>>>>     <configuration>
>>>>        <artifactSet>
>>>>           <includes>
>>>>              <include>com.amazonaws:*</include>
>>>> <include>org.apache.httpcomponents:*</include>
>>>>           </includes>
>>>>        </artifactSet>
>>>>        <relocations>
>>>>           <relocation>
>>>>              <pattern>org.apache.http</pattern>
>>>> <shadedPattern>org.shaded.http</shadedPattern>
>>>>           </relocation>
>>>>        </relocations>
>>>>     </configuration>
>>>>     <executions>
>>>>        <execution>
>>>>           <phase>package</phase>
>>>>           <goals>
>>>>              <goal>shade</goal>
>>>>           </goals>
>>>>        </execution>
>>>>     </executions>
>>>> </plugin>
>>>>
>>>> This seems to work fine and produces a jar that seems to be as expected
>>>> when I build the project using "mvn clean install -Dgpg.skip=true
>>>> -DskipTests=true", although I haven't tested it. Second, I modify the
>>>> aws-java-sdk-s3 pom.xml as follows:
>>>> <plugin>
>>>>     <groupId>org.apache.maven.plugins</groupId>
>>>>     <artifactId>maven-shade-plugin</artifactId>
>>>>     <version>2.4</version>
>>>>     <configuration>
>>>>        <shadeSourcesContent>true</shadeSourcesContent>
>>>>        <createSourcesJar>true</createSourcesJar>
>>>>        <artifactSet>
>>>>           <includes>
>>>>              <include>com.amazonaws:*</include>
>>>> <include>org.apache.httpcomponents:*</include>
>>>>           </includes>
>>>>        </artifactSet>
>>>>        <relocations>
>>>>           <relocation>
>>>>              <pattern>org.apache.http</pattern>
>>>> <shadedPattern>org.shaded.http</shadedPattern>
>>>>           </relocation>
>>>>        </relocations>
>>>>     </configuration>
>>>>     <executions>
>>>>        <execution>
>>>>           <phase>package</phase>
>>>>           <goals>
>>>>              <goal>shade</goal>
>>>>           </goals>
>>>>        </execution>
>>>>     </executions>
>>>> </plugin>
>>>>
>>>> By itself this produces build errors like this:
>>>> [ERROR]
>>>>
>>> \dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\model\S3ObjectInputStream.java:[20,29]
>>>> error: package org.apache.http.client does not exist
>>>>
>>>> So I added an explicit dependency on HTTPComponents to the S3 pom.xml:
>>>> <dependency>
>>>>     <groupId>org.apache.httpcomponents</groupId>
>>>>     <artifactId>httpclient</artifactId>
>>>>     <version>4.3.6</version>
>>>> </dependency>
>>>>
>>>> When I build I get only one error:
>>>> [ERROR]
>>>>
>>> \dev\CSJ\aws\aws-sdk-java-1.10.1\aws-java-sdk-s3\src\main\java\com\amazonaws\services\s3\internal\S3ObjectResponseHandler.java:[54,26]
>>>> error: no suitable constructor found for
>>>>
>>> S3ObjectInputStream(InputStream,org.shaded.http.client.methods.HttpRequestBase)
>>>> It appears to me that the "shading" is not actually relocating the
>>>> http.client.methods.HttpRequestBase argument to the S3ObjectInputStream
>>>> class within the code. Running javap confirms this:
>>>> $ javap
>>>> target/classes/com/amazonaws/services/s3/model/S3ObjectInputStream.class
>>>> Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
>>>> Compiled from "S3ObjectInputStream.java"
>>>> public class com.amazonaws.services.s3.model.S3ObjectInputStream extends
>>>> com.amazonaws.internal.SdkFilterInputStream {
>>>>    public
>>>> com.amazonaws.services.s3.model.S3ObjectInputStream(java.io.InputStream,
>>>> org.apache.http.client.methods.HttpRequestBase);
>>>>    public
>>>> com.amazonaws.services.s3.model.S3ObjectInputStream(java.io.InputStream,
>>>> org.apache.http.client.methods.HttpRequestBase, boolean);
>>>>    public void abort();
>>>>    public org.apache.http.client.methods.HttpRequestBase
>>> getHttpRequest();
>>>>    public int available() throws java.io.IOException;
>>>> }
>>>>
>>>> Can anyone point out if my configuration is wrong, my methodology is
>>>> wrong, or the Maven Shade plugin is actually not behaving as expected?
>>>>
>>>> [1] http://maven.apache.org/plugins/maven-shade-plugin/index.html
>>>> [2] https://github.com/aws/aws-sdk-java
>>>> [3] https://hc.apache.org/
>>>>
>>>> Thanks,
>>>> --
>>>> Michael
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>>> For additional commands, e-mail: users-help@maven.apache.org
>>>>
>>>>
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org


Mime
View raw message