maven-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff MAURY <jeffma...@jeffmaury.com>
Subject Re: Can't manage to download a site with wagon maven plugin
Date Thu, 13 Jun 2013 13:02:57 GMT
I've found the problem with a sample on top of wagon-http. The response is
gzip'ed by the server but the Apache HTTP client or wagon-http does not
unzip the stream -> parsing the response leads to no links detected !!!!

Jeff


On Thu, Jun 13, 2013 at 2:52 PM, Olivier Lamy <olamy@apache.org> wrote:

> not sure you will be able to get that working with current plugins.
> But wagon api has a method to get file list from a directory (
>
> http://maven.apache.org/wagon/wagon-provider-api/apidocs/org/apache/maven/wagon/Wagon.html#getFileList(java.lang.String)
> )
>
> So "easy" to traverse a remote site tree and download all files (but
> IMHO you will need to write your own plugin or improve one)
>
> 2013/6/13 Jeff MAURY <jeffmaury@jeffmaury.com>:
> > Yes, but this should be converted by Maven as it is not possible to
> specify
> > a File in a POM.
> > I'm working with a sample but I think this is related to the returned
> HTML
> > file format.
> >
> > Jeff
> >
> >
> > On Thu, Jun 13, 2013 at 12:12 PM, Martin Gainty <mgainty@hotmail.com>
> wrote:
> >
> >> Jeff download toDir is a file not a string
> >> http://mojo.codehaus.org/wagon-maven-plugin/download-mojo.html
> >>
> >>  try copy
> >> http://mojo.codehaus.org/wagon-maven-plugin/usage.html
> >>
> >> Martin
> >> ______________________________________________
> >> Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité
> >>
> >> Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene
> >> Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede
> unbefugte
> >> Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese
> Nachricht
> >> dient lediglich dem Austausch von Informationen und entfaltet keine
> >> rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von
> >> E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
> >>
> >> Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas
> le
> >> destinataire prévu, nous te demandons avec bonté que pour satisfaire
> >> informez l'expéditeur. N'importe quelle diffusion non autorisée ou la
> copie
> >> de ceci est interdite. Ce message sert à l'information seulement et
> n'aura
> >> pas n'importe quel effet légalement obligatoire. Étant donné que les
> email
> >> peuvent facilement être sujets à la manipulation, nous ne pouvons
> accepter
> >> aucune responsabilité pour le contenu fourni.
> >>
> >>
> >> > From: jeffmaury@jeffmaury.com
> >> > Date: Thu, 13 Jun 2013 10:55:39 +0200
> >> > Subject: Can't manage to download a site with wagon maven plugin
> >> > To: users@maven.apache.org
> >> >
> >> > Hello,
> >> >
> >> > I want to setup my pom in order to dowload a remote web site through
> >> HTTP.
> >> > Here is my POM:
> >> >
> >> > <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="
> >> > http://www.w3.org/2001/XMLSchema-instance"
> >> >  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> >> > http://maven.apache.org/xsd/maven-4.0.0.xsd">
> >> >  <modelVersion>4.0.0</modelVersion>
> >> >
> >> > <groupId>com.jeffmaury</groupId>
> >> >  <artifactId>wagon-http-test</artifactId>
> >> > <version>0.0.1-SNAPSHOT</version>
> >> > <packaging>jar</packaging>
> >> >
> >> > <name>wagon-http-test</name>
> >> > <url>http://maven.apache.org</url>
> >> >
> >> > <properties>
> >> > <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
> >> >  </properties>
> >> >
> >> > <build>
> >> > <extensions>
> >> >  <extension>
> >> > <groupId>org.apache.maven.wagon</groupId>
> >> > <artifactId>wagon-http</artifactId>
> >> >  <version>2.4</version>
> >> > </extension>
> >> > </extensions>
> >> >  <plugins>
> >> > <plugin>
> >> > <groupId>org.codehaus.mojo</groupId>
> >> >  <artifactId>wagon-maven-plugin</artifactId>
> >> > <version>1.0-beta-4</version>
> >> >  <executions>
> >> > <execution>
> >> > <phase>generate-resources</phase>
> >> >  <goals>
> >> > <goal>download</goal>
> >> > </goals>
> >> >  </execution>
> >> > </executions>
> >> > <configuration>
> >> >  <url>http://www.syspertec.com</url>
> >> > <fromDir>/</fromDir>
> >> >  <toDir>${project.outputDirectory}/test</toDir>
> >> > <includes>**/*</includes>
> >> >  </configuration>
> >> > </plugin>
> >> > </plugins>
> >> >  </build>
> >> > </project>
> >> >
> >> > Here is the output:
> >> >
> >> > [INFO] Scanning for projects...
> >> > [INFO]
> >> > [INFO]
> >> >
> ------------------------------------------------------------------------
> >> > [INFO] Building wagon-http-test 0.0.1-SNAPSHOT
> >> > [INFO]
> >> >
> ------------------------------------------------------------------------
> >> > [INFO]
> >> > *[INFO] --- wagon-maven-plugin:1.0-beta-4:download (default) @
> >> > wagon-http-test ---*
> >> > *[INFO] Scanning remote file system: http://www.syspertec.com ...*
> >> > *[INFO] Nothing to download.*
> >> > [INFO]
> >> > [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @
> >> > wagon-http-test ---
> >> > [debug] execute contextualize
> >> > [INFO] Using 'UTF-8' encoding to copy filtered resources.
> >> > [INFO] skip non existing resourceDirectory
> >> > d:\workspace\wagon-http-test\src\main\resources
> >> > [INFO]
> >> > [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @
> >> > wagon-http-test ---
> >> > [INFO] Nothing to compile - all classes are up to date
> >> > [INFO]
> >> >
> ------------------------------------------------------------------------
> >> > [INFO] BUILD SUCCESS
> >> > [INFO]
> >> >
> ------------------------------------------------------------------------
> >> > [INFO] Total time: 3.162s
> >> > [INFO] Finished at: Thu Jun 13 10:53:50 CEST 2013
> >> > [INFO] Final Memory: 6M/73M
> >> > [INFO]
> >> >
> ------------------------------------------------------------------------
> >> > /d/workspace/wagon-http-test
> >> >
> >> > Any help ?
> >> > --
> >> > Jeff MAURY
> >> > PS; I'm posting to Maven user list instead of Mojo because I suspect
> this
> >> > is related to wagon-http
> >> >
> >> >
> >> > "Legacy code" often differs from its suggested alternative by actually
> >> > working and scaling.
> >> >  - Bjarne Stroustrup
> >> >
> >> > http://www.jeffmaury.com
> >> > http://riadiscuss.jeffmaury.com
> >> > http://www.twitter.com/jeffmaury
> >>
> >>
> >
> >
> >
> > --
> > Jeff MAURY
> >
> >
> > "Legacy code" often differs from its suggested alternative by actually
> > working and scaling.
> >  - Bjarne Stroustrup
> >
> > http://www.jeffmaury.com
> > http://riadiscuss.jeffmaury.com
> > http://www.twitter.com/jeffmaury
>
>
>
> --
> Olivier Lamy
> Ecetera: http://ecetera.com.au
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>


-- 
Jeff MAURY


"Legacy code" often differs from its suggested alternative by actually
working and scaling.
 - Bjarne Stroustrup

http://www.jeffmaury.com
http://riadiscuss.jeffmaury.com
http://www.twitter.com/jeffmaury

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message