hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom White <...@cloudera.com>
Subject Re: follow up Hadoop mavenization work
Date Tue, 02 Aug 2011 17:37:12 GMT
HADOOP-6671 is now in trunk. After you do an update you will need to
use Maven 3 to build Common. There are instructions at


HDFS and MapReduce still use Ant, and the instructions for
cross-project builds are in

Please send any questions/ideas for improvements, etc to the dev list.


On Tue, Aug 2, 2011 at 9:16 AM, Tom White <tom@cloudera.com> wrote:
> On Sun, Jul 31, 2011 at 9:57 PM, Tom White <tom@cloudera.com> wrote:
>> HADOOP-6671 has now received two +1s (one from Eric Yang and one from
>> me), so I would like to commit it on Tuesday at 16:00 GMT
>> (http://s.apache.org/6nx). I'll also update the Jenkins jobs for
>> running test-patch and performing the nightly build.
> I'm going to go ahead and commit the patch now.
> Tom
>> For developers this change will mean that you need to use Maven to
>> build Hadoop Common. The build instructions are listed in the
>> BUILDING.txt file in the patch, as well as at
>> http://wiki.apache.org/hadoop/HowToContribute and
>> http://s.apache.org/wb.
>> Note that HDFS and MapReduce will still use Ant for building, but
>> follow on JIRAs HDFS-2096 and MAPREDUCE-2607 will introduce Maven to
>> those builds in the near future. (In terms of staging, it makes sense
>> for MAPREDUCE-2607 to go in after MAPREDUCE-279, since the MR2 work
>> uses Maven to build its new modules, so the Mavenization of MapReduce
>> should build on that work.)
>> Thanks,
>> Tom
>> On Thu, Jul 28, 2011 at 4:41 PM, Alejandro Abdelnur <tucu@cloudera.com> wrote:
>>> Following up with Hadoop Common mavenization (HADOOP-6671) I've just posted
>>> a patch for HDFS mavenization (HDFS-2096)
>>> The HADOOP-6671 patch integrates all feedback received in the JIRA and, IMO,
>>> it is ready for prime time.
>>> In order not break HDFS and MAPRED which are still Ant based, there are 2
>>> patches HDFS-2196 & MAPREDUCE-2741that make some corrections in the ivy
>>> configuration to work correctly with the Hadoop common JAR (build/published
>>> by Mavenized build).
>>> HDFS-2096 is not 100% ready, some testcases are failing and native code
>>> testing is not wired, but everything else (compile, test, package, tar,
>>> binary, jdiff, etc is wired).
>>> * https://issues.apache.org/jira/browse/HADOOP-6671
>>> * https://issues.apache.org/jira/browse/HDFS-2196
>>> * https://issues.apache.org/jira/browse/MAPREDUCE-2741
>>> * https://issues.apache.org/jira/browse/HDFS-2096
>>> I know these are big changes and we'll have some hiccups, but the benefits
>>> are big (running testcases is faster, it easily works from IDEs, Maven build
>>> system can easily be understood by anybody that knows Maven).
>>> Keeping the patches current is time-consuming, because of this, it would be
>>> great if we can get in the ones ready (HADOOP-6671, HDFS-2196,
>>> MAPREDUCE-2741) so we can focus on the rest of the Mavenization work.
>>> Thanks.
>>> Alejandro

View raw message