ant-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maarten Coene (JIRA)" <j...@apache.org>
Subject [jira] Commented: (IVY-1194) Downloads from maven repository fail when using transparent HTTP proxies
Date Thu, 10 Jun 2010 20:13:15 GMT

    [ https://issues.apache.org/jira/browse/IVY-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877553#action_12877553
] 

Maarten Coene commented on IVY-1194:
------------------------------------

FYI: I've add support for "deflate" as well.
I did refactor the code a bit for this, could you give it another try to see I 'gzip' still
works for you?

thanks
Maarten

> Downloads from maven repository fail when using transparent HTTP proxies
> ------------------------------------------------------------------------
>
>                 Key: IVY-1194
>                 URL: https://issues.apache.org/jira/browse/IVY-1194
>             Project: Ivy
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 2.1.0
>         Environment: Windows XP, Java 1.5.0.16, java 1.6.0.20
>            Reporter: Constantine Plotnikov
>            Assignee: Maarten Coene
>            Priority: Blocker
>             Fix For: 2.2.0-RC1
>
>
> The package without dependencies cannot fetch dependencies and samples in the package
with dependencies do not work.
> Fetching dependencies fails with the following log:
> {quote}
> Buildfile: build.xml
> init-ivy:
> retrieve-all:
> [ivy:retrieve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
> [ivy:retrieve] :: loading settings :: url = jar:file:/C:/utils/apache/apache-ivy-2.1.0-wo/ivy.jar!/org/apache/ivy/core/settings/ivysettings.xml
> [ivy:retrieve] :: resolving dependencies :: org.apache.ivy#ivy;2.1.0
> [ivy:retrieve] 	confs: [core, httpclient, oro, vfs, sftp, standalone, ant, default, test,
source]
> [ivy:retrieve] 	found commons-httpclient#commons-httpclient;3.0 in public
> [ivy:retrieve] 	found commons-codec#commons-codec;1.2 in public
> [ivy:retrieve] 	found oro#oro;2.0.8 in public
> [ivy:retrieve] 	found commons-vfs#commons-vfs;1.0 in public
> [ivy:retrieve] 	found com.jcraft#jsch;0.1.31 in public
> [ivy:retrieve] 	found ant#ant;1.6.2 in public
> [ivy:retrieve] 	found ant#ant-nodeps;1.6.2 in public
> [ivy:retrieve] 	found ant#ant-trax;1.6.2 in public
> [ivy:retrieve] 	found junit#junit;3.8.2 in public
> [ivy:retrieve] 	found commons-lang#commons-lang;2.4 in public
> [ivy:retrieve] 	found org.apache.ant#ant-testutil;1.7.0 in public
> [ivy:retrieve] 	found ant#ant-launcher;1.6.2 in public
> [ivy:retrieve] 	found xerces#xercesImpl;2.6.2 in public
> [ivy:retrieve] 	found xerces#xmlParserAPIs;2.6.2 in public
> [ivy:retrieve] downloading http://repo1.maven.org/maven2/junit/junit/3.8.2/junit-3.8.2.jar
...
> [ivy:retrieve] ............ (117kB)
> [ivy:retrieve] .. (0kB)
> [ivy:retrieve] :: resolution report :: resolve 1250ms :: artifacts dl 250ms
> [ivy:retrieve] 	:: evicted modules:
> [ivy:retrieve] 	commons-logging#commons-logging;1.0.3 by [commons-logging#commons-logging;1.0.4]
in [default]
> 	---------------------------------------------------------------------
> 	|                  |            modules            ||   artifacts   |
> 	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
> 	---------------------------------------------------------------------
> 	|       core       |   0   |   0   |   0   |   0   ||   0   |   0   |
> 	|    httpclient    |   3   |   0   |   0   |   0   ||   2   |   0   |
> 	|        oro       |   1   |   0   |   0   |   0   ||   1   |   0   |
> 	|        vfs       |   2   |   0   |   0   |   0   ||   1   |   0   |
> 	|       sftp       |   1   |   0   |   0   |   0   ||   1   |   0   |
> 	|    standalone    |   0   |   0   |   0   |   0   ||   0   |   0   |
> 	|        ant       |   1   |   0   |   0   |   0   ||   1   |   0   |
> 	|      default     |   10  |   0   |   0   |   1   ||   8   |   0   |
> 	|       test       |   6   |   0   |   0   |   0   ||   6   |   0   |
> 	|      source      |   0   |   0   |   0   |   0   ||   0   |   0   |
> 	---------------------------------------------------------------------
> [ivy:retrieve] :: problems summary ::
> [ivy:retrieve] :::: WARNINGS
> [ivy:retrieve] 	problem while downloading module descriptor: http://repo1.maven.org/maven2/commons-logging/commons-logging/1.0.3/commons-logging-1.0.3.pom:
invalid sha1: expected=<
> [ivy:retrieve] г computed=88c58ea4a562116ab15fb76c9097ee1f25cc750b (62ms)
> [ivy:retrieve] 		module not found: commons-logging#commons-logging;1.0.3
> [ivy:retrieve] 	==== local: tried
> [ivy:retrieve] 	  C:\Documents and Settings\Constantin.Plotnikov\.ivy2/local/commons-logging/commons-logging/1.0.3/ivys/ivy.xml
> [ivy:retrieve] 	  -- artifact commons-logging#commons-logging;1.0.3!commons-logging.jar:
> [ivy:retrieve] 	  C:\Documents and Settings\Constantin.Plotnikov\.ivy2/local/commons-logging/commons-logging/1.0.3/jars/commons-logging.jar
> [ivy:retrieve] 	==== shared: tried
> [ivy:retrieve] 	  C:\Documents and Settings\Constantin.Plotnikov\.ivy2/shared/commons-logging/commons-logging/1.0.3/ivys/ivy.xml
> [ivy:retrieve] 	  -- artifact commons-logging#commons-logging;1.0.3!commons-logging.jar:
> [ivy:retrieve] 	  C:\Documents and Settings\Constantin.Plotnikov\.ivy2/shared/commons-logging/commons-logging/1.0.3/jars/commons-logging.jar
> [ivy:retrieve] 	==== public: tried
> [ivy:retrieve] 	  http://repo1.maven.org/maven2/commons-logging/commons-logging/1.0.3/commons-logging-1.0.3.pom
> [ivy:retrieve] 	problem while downloading module descriptor: http://repo1.maven.org/maven2/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.pom:
invalid sha1: expected=<
> [ivy:retrieve] 		module not found: commons-logging#commons-logging;1.0.4
> [ivy:retrieve] 	==== local: tried
> [ivy:retrieve] 	  C:\Documents and Settings\Constantin.Plotnikov\.ivy2/local/commons-logging/commons-logging/1.0.4/ivys/ivy.xml
> [ivy:retrieve] 	  -- artifact commons-logging#commons-logging;1.0.4!commons-logging.jar:
> [ivy:retrieve] 	  C:\Documents and Settings\Constantin.Plotnikov\.ivy2/local/commons-logging/commons-logging/1.0.4/jars/commons-logging.jar
> [ivy:retrieve] 	==== shared: tried
> [ivy:retrieve] 	  C:\Documents and Settings\Constantin.Plotnikov\.ivy2/shared/commons-logging/commons-logging/1.0.4/ivys/ivy.xml
> [ivy:retrieve] 	  -- artifact commons-logging#commons-logging;1.0.4!commons-logging.jar:
> [ivy:retrieve] 	  C:\Documents and Settings\Constantin.Plotnikov\.ivy2/shared/commons-logging/commons-logging/1.0.4/jars/commons-logging.jar
> [ivy:retrieve] 	==== public: tried
> [ivy:retrieve] 	  http://repo1.maven.org/maven2/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.pom
> [ivy:retrieve] 		[FAILED     ] junit#junit;3.8.2!junit.jar: invalid sha1: expected=<
> [ivy:retrieve] 		[FAILED     ] junit#junit;3.8.2!junit.jar:  (0ms)
> [ivy:retrieve] 	==== shared: tried
> [ivy:retrieve] 	  C:\Documents and Settings\Constantin.Plotnikov\.ivy2/shared/junit/junit/3.8.2/jars/junit.jar
> [ivy:retrieve] 	==== public: tried
> [ivy:retrieve] 	  http://repo1.maven.org/maven2/junit/junit/3.8.2/junit-3.8.2.jar
> [ivy:retrieve] 		::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] 		::          UNRESOLVED DEPENDENCIES         ::
> [ivy:retrieve] 		::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] 		:: commons-logging#commons-logging;1.0.3: not found
> [ivy:retrieve] 		:: commons-logging#commons-logging;1.0.4: not found
> [ivy:retrieve] 		::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] 
> [ivy:retrieve] 		::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] 		::              FAILED DOWNLOADS            ::
> [ivy:retrieve] 		:: ^ see resolution messages for details  ^ ::
> [ivy:retrieve] 		::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] 		:: junit#junit;3.8.2!junit.jar
> [ivy:retrieve] 		::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] 
> [ivy:retrieve] 
> [ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
> {quote}
> The sample in package with dependencies  fails with the following log:
> {quote}
> Buildfile: build.xml
> download-ivy:
>     [mkdir] Created dir: C:\utils\apache\apache-ivy-2.1.0\src\example\go-ivy\ivy
>      [echo] installing ivy...
>       [get] Getting: http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
>       [get] To: C:\utils\apache\apache-ivy-2.1.0\src\example\go-ivy\ivy\ivy.jar
> install-ivy:
>   [taskdef] java.util.zip.ZipException: error in opening zip file
>   [taskdef] 	at java.util.zip.ZipFile.open(Native Method)
>   [taskdef] 	at java.util.zip.ZipFile.<init>(ZipFile.java:203)
>   [taskdef] 	at java.util.zip.ZipFile.<init>(ZipFile.java:234)
>   [taskdef] 	at org.apache.tools.ant.AntClassLoader.getResourceURL(AntClassLoader.java:919)
>   [taskdef] 	at org.apache.tools.ant.AntClassLoader$ResourceEnumeration.findNextResource(AntClassLoader.java:126)
>   [taskdef] 	at org.apache.tools.ant.AntClassLoader$ResourceEnumeration.<init>(AntClassLoader.java:88)
>   [taskdef] 	at org.apache.tools.ant.AntClassLoader.findResources(AntClassLoader.java:869)
>   [taskdef] 	at java.lang.ClassLoader.getResources(ClassLoader.java:1015)
>   [taskdef] 	at org.apache.tools.ant.taskdefs.Definer.resourceToURLs(Definer.java:267)
>   [taskdef] 	at org.apache.tools.ant.taskdefs.Definer.execute(Definer.java:211)
>   [taskdef] 	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275)
>   [taskdef] 	at org.apache.tools.ant.Task.perform(Task.java:364)
>   [taskdef] 	at org.apache.tools.ant.Target.execute(Target.java:341)
>   [taskdef] 	at org.apache.tools.ant.Target.performTasks(Target.java:369)
>   [taskdef] 	at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1216)
>   [taskdef] 	at org.apache.tools.ant.Project.executeTarget(Project.java:1185)
>   [taskdef] 	at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:40)
>   [taskdef] 	at org.apache.tools.ant.Project.executeTargets(Project.java:1068)
>   [taskdef] 	at org.apache.tools.ant.Main.runBuild(Main.java:668)
>   [taskdef] 	at org.apache.tools.ant.Main.startAnt(Main.java:187)
>   [taskdef] 	at org.apache.tools.ant.launch.Launcher.run(Launcher.java:246)
>   [taskdef] 	at org.apache.tools.ant.launch.Launcher.main(Launcher.java:67)
>   [taskdef] Could not load definitions from resource org/apache/ivy/ant/antlib.xml. It
could not be found.
> generate-src:
> go:
>      [echo] using ivy to resolve commons-lang 2.1...
> BUILD FAILED
> C:\utils\apache\apache-ivy-2.1.0\src\example\go-ivy\build.xml:85: Could not create task
or type of type: antlib:org.apache.ivy.ant:cachepath.
> Ant could not find the task or a class this task relies upon.
> This is common and has a number of causes; the usual 
> solutions are to read the manual pages then download and
> install needed JAR files, or fix the build file: 
>  - You have misspelt 'antlib:org.apache.ivy.ant:cachepath'.
>    Fix: check your spelling.
>  - The task needs an external JAR file to execute
>      and this is not found at the right place in the classpath.
>    Fix: check the documentation for dependencies.
>    Fix: declare the task.
>  - The task is an Ant optional task and the JAR file and/or libraries
>      implementing the functionality were not found at the time you
>      yourself built your installation of Ant from the Ant sources.
>    Fix: Look in the ANT_HOME/lib for the 'ant-' JAR corresponding to the
>      task and make sure it contains more than merely a META-INF/MANIFEST.MF.
>      If all it contains is the manifest, then rebuild Ant with the needed
>      libraries present in ${ant.home}/lib/optional/ , or alternatively,
>      download a pre-built release version from apache.org
>  - The build file was written for a later version of Ant
>    Fix: upgrade to at least the latest release version of Ant
>  - The task is not an Ant core or optional task 
>      and needs to be declared using <taskdef>.
>  - You are attempting to use a task defined using 
>     <presetdef> or <macrodef> but have spelt wrong or not 
>    defined it at the point of use
> Remember that for JAR files to be visible to Ant tasks implemented
> in ANT_HOME/lib, the files must be in the same directory or on the
> classpath
> Please neither file bug reports on this problem, nor email the
> Ant mailing lists, until all of these causes have been explored,
> as this is not an Ant bug.
> Total time: 1 second
> {quote}
> The reason for the bug is the squid. When request comes and it has a cached content with
gzip content encoding, it return a content with encoded content, event if no encoding was
specified in the request or "Accept-Encoding: identity" was specified. The below is a sample
log that demonstrates the problem:  
> {quote}
> DEBUG output created by Wget 1.11.4 on cygwin.
> --2010-06-02 12:41:55--  http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
> Resolving repo1.maven.org... 38.97.124.18
> Caching repo1.maven.org => 38.97.124.18
> Connecting to repo1.maven.org|38.97.124.18|:80... connected.
> Created socket 4.
> Releasing 0x006d9100 (new refcount 1).
> ---request begin---
> GET /maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar HTTP/1.0
> User-Agent: Wget/1.11.4
> Accept: */*
> Host: repo1.maven.org
> Connection: Keep-Alive
> ---request end---
> HTTP request sent, awaiting response... 
> ---response begin---
> HTTP/1.0 200 OK
> Server: nginx/0.7.62
> Date: Tue, 01 Jun 2010 13:09:36 GMT
> Content-Type: application/java-archive
> Last-Modified: Mon, 12 Oct 2009 21:19:02 GMT
> Content-Encoding: gzip
> X-Cache: MISS from squid_havp_node1
> X-Cache-Lookup: MISS from squid_havp_node1:3128
> Age: 70339
> Content-Length: 834513
> X-Cache: HIT from squid_havp_node1
> X-Cache-Lookup: HIT from squid_havp_node1:3128
> Connection: keep-alive
> ---response end---
>   HTTP/1.0 200 OK
>   Server: nginx/0.7.62
>   Date: Tue, 01 Jun 2010 13:09:36 GMT
>   Content-Type: application/java-archive
>   Last-Modified: Mon, 12 Oct 2009 21:19:02 GMT
>   Content-Encoding: gzip
>   X-Cache: MISS from squid_havp_node1
>   X-Cache-Lookup: MISS from squid_havp_node1:3128
>   Age: 70339
>   Content-Length: 834513
>   X-Cache: HIT from squid_havp_node1
>   X-Cache-Lookup: HIT from squid_havp_node1:3128
>   Connection: keep-alive
> Registered socket 4 for persistent reuse.
> Length: 834513 (815K) [application/java-archive]
> Saving to: `ivy-2.1.0.jar'
>      0K .......... .......... .......... .......... ..........  6% 8.49M 0s
>     50K .......... .......... .......... .......... .......... 12% 12.0M 0s
>    100K .......... .......... .......... .......... .......... 18% 7.26M 0s
>    150K .......... .......... .......... .......... .......... 24% 9.28M 0s
>    200K .......... .......... .......... .......... .......... 30% 5.33M 0s
>    250K .......... .......... .......... .......... .......... 36% 9.31M 0s
>    300K .......... .......... .......... .......... .......... 42% 5.26M 0s
>    350K .......... .......... .......... .......... .......... 49% 9.15M 0s
>    400K .......... .......... .......... .......... .......... 55% 7.27M 0s
>    450K .......... .......... .......... .......... .......... 61% 7.42M 0s
>    500K .......... .......... .......... .......... .......... 67% 9.21M 0s
>    550K .......... .......... .......... .......... .......... 73% 6.11M 0s
>    600K .......... .......... .......... .......... .......... 79% 9.50M 0s
>    650K .......... .......... .......... .......... .......... 85% 8.84M 0s
>    700K .......... .......... .......... .......... .......... 92% 9.54M 0s
>    750K .......... .......... .......... .......... .......... 98% 6.04M 0s
>    800K .......... ....                                       100% 16.2K=1.0s
> 2010-06-02 12:41:56 (795 KB/s) - `ivy-2.1.0.jar' saved [834513/834513]
> {quote}
> In our environment, a bunch of files from repo1.maven.org are cached in the squid. So
there were a lot of conflicts with content encoding.
> The possible workarounds:
> 1. Check returned content encoding from http client, if it is not identity, reissue the
request with disabling the caches.
> 2. Ungzip files fetched with gzip content encoding.
> 3. Support gzip content encoding in common HTTP.
> 3. Use local maven proxies 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message