hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Hadoop JARs and Eclipse
Date Thu, 06 Jun 2013 02:18:22 GMT
Hi John,

On Thu, Jun 6, 2013 at 1:21 AM, John Lilley <john.lilley@redpoint.net> wrote:
> -- From where will it fetch the Hadoop JARs?

>From the Maven Central repository (we publish our jars and
dependencies are also available there), or a custom defined repository
if you lack internet access.

> -- If I have built Hadoop from source, how do I tell it to find my just-built JARs instead
of those from the official release site?

If you built Hadoop locally with "mvn install", then their version is
added to your local repository (~/.m2) and can be "referred' to after
a successful "mvn install".

> -- Is it necessary to build Hadoop from source so that the Eclipse debugger can locate
source files and display them if I need to step in?

No, see below.

> -- Step into the source code of referenced Hadoop code in the debugger.

This one is easy with the Eclipse Maven IDE plugin, see
This lets you view and also debug-step into sources of dependencies.

> -- Make changes to the Hadoop source and rebuild it if things are not working as expected
(e.g. to add logging).
> -- Switch to building from SVN instead of a release snapshot, and apply patches.

If you're wanting to use the branch-2 tip or the trunk tip, then they
carry their own version (2.1.0-beta-SNAPSHOT, or 3.0.0-SNAPSHOT and
such). Referring these versions in the pom.xml of your project would
be the first step (you may also have to enable discovery of snapshot
releases?). Second would be to make the changes to the hadoop source
and running a global "mvn install" to update the local repository
(~/.m2) versions to the patched ones. Subsequently, your project will
be able to find it.

What am saying is you can do this without having to manage Hadoop as
an eclipse project - just patch and build (plus install) on CLI, but
mark your project to reference their exact SNAPSHOT version so it
discovers the new built jars.

Harsh J

View raw message