incubator-giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gianmarco De Francisci Morales <g...@apache.org>
Subject Re: Packaging a Giraph application in a jar
Date Thu, 10 Nov 2011 15:31:07 GMT
Nice!
It looks cleaner than my ant+ivy hack :)

Thanks for sharing,
--
Gianmarco



On Thu, Nov 10, 2011 at 16:18, Claudio Martella
<claudio.martella@gmail.com>wrote:

> Ok,
>
> quite basically this is my pom.xml:
>
> <project xmlns="http://maven.apache.org/POM/4.0.0"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> http://maven.apache.org/maven-v4_0_0.xsd">
>        <modelVersion>4.0.0</modelVersion>
>        <groupId>org.acaro.graffiti</groupId>
>        <artifactId>graffiti</artifactId>
>        <packaging>jar</packaging>
>        <version>1.0-SNAPSHOT</version>
>        <name>graffiti</name>
>        <url>http://maven.apache.org</url>
>        <build>
>                <plugins>
>                        <plugin>
>                                <groupId>org.apache.maven.plugins</groupId>
>
>  <artifactId>maven-assembly-plugin</artifactId>
>                                <version>2.2</version>
>                                <configuration>
>
>  <outputDirectory>target</outputDirectory>
>                                        <descriptorRefs>
>
>  <descriptorRef>jar-with-dependencies</descriptorRef>
>                                        </descriptorRefs>
>                                </configuration>
>                                <executions>
>                                        <execution>
>                                                <id>make-assembly</id>
>                                                <phase>compile</phase>
>                                                <goals>
>                                                        <goal>single</goal>
>                                                </goals>
>                                        </execution>
>                                </executions>
>                        </plugin>
>                        <plugin>
>                                <groupId>org.apache.maven.plugins</groupId>
>
>  <artifactId>maven-eclipse-plugin</artifactId>
>                                <configuration>
>
>  <downloadSources>true</downloadSources>
>
>  <downloadJavadocs>true</downloadJavadocs>
>                                </configuration>
>                        </plugin>
>                        <plugin>
>                                <groupId>org.apache.maven.plugins</groupId>
>                                <artifactId>maven-shade-plugin</artifactId>
>                                <version>1.5</version>
>                                <executions>
>                                        <execution>
>                                                <phase>package</phase>
>                                                <goals>
>                                                        <goal>shade</goal>
>                                                </goals>
>                                                <configuration>
>                                                </configuration>
>                                        </execution>
>                                </executions>
>                        </plugin>
>                </plugins>
>        </build>
>        <dependencies>
>                <dependency>
>                        <groupId>junit</groupId>
>                        <artifactId>junit</artifactId>
>                        <version>4.8.2</version>
>                        <scope>test</scope>
>                </dependency>
>                <dependency>
>                        <groupId>ch.qos.logback</groupId>
>                        <artifactId>logback-classic</artifactId>
>                        <version>0.9.27</version>
>                </dependency>
>                <dependency>
>                        <groupId>com.google.guava</groupId>
>                        <artifactId>guava</artifactId>
>                        <version>r09</version>
>                </dependency>
>                <dependency>
>                        <groupId>org.antlr</groupId>
>                        <artifactId>antlr</artifactId>
>                        <version>3.3</version>
>                </dependency>
>                <dependency>
>                        <groupId>org.apache.giraph</groupId>
>                        <artifactId>giraph</artifactId>
>                        <version>0.70</version>
>                </dependency>
>                <dependency>
>                        <groupId>org.apache.hadoop</groupId>
>                        <artifactId>hadoop-core</artifactId>
>                        <version>0.20.204.0</version>
>                        <scope>provided</scope>
>                </dependency>
>        </dependencies>
> </project>
>
>
> You set your project dependecies as usual, set hadoop-core (you don't
> really need it in the job jar) as <provided> and then use the
> maven-shade (http://maven.apache.org/plugins/maven-shade-plugin/). At
> package goal it will upack your dependencies and re-pack them along
> with your project classes in an uberjar.
>
> maven-shade allows you to set excludes in case of conflicts.
>
> Easy, automatic and clean.
>
>
> Hope this helps,
> Claudio
>
> On Wed, Nov 9, 2011 at 12:19 PM, Avery Ching <aching@apache.org> wrote:
> > Would be great if you can document what you did. =)
> >
> > Thanks,
> >
> > Avery
> >
> > On 11/8/11 3:13 PM, Claudio Martella wrote:
> >>
> >> Sorry guys, may bad.
> >>
> >> Was calling job.waitForCompletion() directly. I've been coding
> >> standard mapreduce whole weekend...
> >>
> >> Anyway I got a solution for clean packaging of your own application
> >> over giraph, and that is exactly using maven-shade-plugin. it will
> >> prepare the uberjar for you.
> >>
> >> On Tue, Nov 8, 2011 at 9:33 PM, Claudio Martella
> >> <claudio.martella@gmail.com>  wrote:
> >>>
> >>> Hello list,
> >>>
> >>> I'm actually having troubles as well to get my application running.
> >>>
> >>> I've give a shot to maven-shade plugin which unpacks my dependencies
> >>> and packs them all together with my classes in a new jar.
> >>>
> >>> I attach the hierarchy of the jar so that somebody can maybe spot
> >>> what's missing, because i can't get it working. I get an identity
> >>> map-reduce job with jobconf complaining about no job jar being set.
> >>>
> >>> Any idea?
> >>>
> >>> On Sat, Nov 5, 2011 at 5:09 PM, Avery Ching<aching@apache.org>  wrote:
> >>>>
> >>>> Hi Gianmarco,
> >>>>
> >>>> You're right, most of us (to my knowledge) have been using Giraph with
> >>>> an
> >>>> uberjar as you've put it.  However, Jakob has been doing some work to
> >>>> make
> >>>> this easier.  See the below issue:
> >>>>
> >>>> https://issues.apache.org/jira/browse/GIRAPH-64
> >>>>
> >>>> If you can suggest a better approach, please add to the issue or
> create
> >>>> a
> >>>> new one if appropriate.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Avery
> >>>>
> >>>> On 11/5/11 4:11 AM, Gianmarco De Francisci Morales wrote:
> >>>>>
> >>>>> Hi community,
> >>>>>
> >>>>> I was wondering what is the current best practice to package an
> >>>>> application in a jar for deployment.
> >>>>> I tried the 'hadoop way' by putting giraph-*.jar in the /lib
> directory
> >>>>> of
> >>>>> my jar, and using the -libjars option but none of them worked. It
> looks
> >>>>> like
> >>>>> the backend classloader is doing some mess and it doesn't find my
own
> >>>>> classes in the jar.
> >>>>>
> >>>>> I resorted to uncompressing the giraph-*.jar and repackaging my
> classes
> >>>>> with it, all at the same level (an uber-fat jar), but even though
it
> >>>>> works
> >>>>> it doesn't sound like the right approach.
> >>>>>
> >>>>> Any suggestions?
> >>>>>
> >>>>> Thanks,
> >>>>> --
> >>>>> Gianmarco
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>> --
> >>>     Claudio Martella
> >>>     claudio.martella@gmail.com
> >>>
> >>
> >>
> >
> >
>
>
>
> --
>     Claudio Martella
>     claudio.martella@gmail.com
>

Mime
View raw message