hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Kimball <aa...@cloudera.com>
Subject Re: How to build and deploy Hadoop 0.21 ?
Date Mon, 09 Nov 2009 04:13:47 GMT
On Thu, Nov 5, 2009 at 2:34 AM, Andrei Dragomir <adragomi@adobe.com> wrote:

> Hello everyone.
> We ran into a bunch of issues with building and deploying hadoop 0.21.
> It would be great to get some answers about how things should work, so
> we can try to fix them.
> 1. When checking out the repositories, each of them can be built by
> itself perfectly. BUT, if you look in hdfs it has mapreduce libraries,
> and in mapreduce it has hdfs libraries. That's kind of a cross-
> reference between projects.
>        Q: Is this dependence necessary ? Can we get rid of it ?

Those are build-time dependencies. Ideally you'll ignore them post-build.

>        Q: if it's necessary, how does one build the jars with the latest
> version of the source code ? how are the jars in the scm repository
> created  (hadoop-hdfs/lib/hadoop-mapred-0.21-dev.jar) as long as there
> is a cross-reference ?
> 2. There are issues with the jar files and the webapps (dfshealth.jsp,
> etc). Right now, the only way to have a hadoop functioning system is
> to: build hdfs and mapreduce; copy everything from hdfs/build and
> mapreduce/build to common/build.


>        Q: Is there a better way of doing this ? What needs to be fixed to
> have the webapps in the jar files (like on 0.20). Are there JIRA
> issues logged on this ?
I have created a Makefile and some associated scripts that will build
everything and squash it together for you; see

There is also a longer-term effort to use Maven to coordinate the three
subprojects, and use a local repository for inter-project development on a
single machine; see https://issues.apache.org/jira/browse/HADOOP-5107 for
progress there.

> We would really appreciate some answers at least related to where
> hadoop is going with this build step, so we can help with patches /
> fixes.
> Thank you,
>   Andrei Dragomirt

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message