hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14453) Split the maven modules into several profiles
Date Wed, 31 May 2017 13:57:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16031191#comment-16031191
] 

Steve Loughran commented on HADOOP-14453:
-----------------------------------------

Probably:
{code}
mvn -T 1C -DskipShade -Dtest=TestReadWhileWriting --pl hadoop-hdfs-project/hdfs-client --pl
hadoop-hdfs-project/hadoop-hdfs
{code}
If the test case isn't found in a specific module, the build continues, so you can list >1
module and still only run the single test case (though as it does still start the surefire
test run for all modules, there's still setup time). Test fail: immediate halt unless you
set {{--fase}}. Test succeed: continue through the list of projects, running any other test
cases with that name.


FWIW. my build workflow is 
* use {{fish}} as my shell, with [macros for the common maven commands|https://github.com/steveloughran/clusterconfigs/tree/master/bin/functions]
* different tabs in a terminal for different sub modules (today hadoop-common, hadoop-mapreduce,
hadoop-aws)
* and the root one for the overall project, which takes ~5 minutes first thing in the morning,
or when I want a full clean build.

I also, believe it or not, use the IDE, specifically IntelliJ IDEA with the maven projects
all imported. 

To run a test here I: 
* open the test suite class
* find the test case I want to run
* set any breakpoints I want
* select the line declaring the test method, and then the little popup helper menu
* and "debug this test case"

First run of the day, yes, build. After that, 20s launch time. Faster than maven, lets me
debug what is going on without adding lots of log statements. Encourages me to add toString()
methods to classes so the the view of variable state is meaningful, lets me step through line-by-line,
lets me set breakpoints on various branches to verify code coverage.

> Split the maven modules into several profiles
> ---------------------------------------------
>
>                 Key: HADOOP-14453
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14453
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: build
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>         Attachments: c14453_20170524.patch
>
>
> Current all the modules are defined at directly under <project>.  As a result,
we cannot select to build only some of the modules.  We have to build all the modules in any
cases and, unfortunately, it takes a long time.
> We propose split all the modules into multiple profiles so that we could build some of
the modules by disabling some of the profiles.  All the profiles are enabled by default so
that all the modules will be built by default. 
> For example, when we are making change in common.  We could build and run tests under
common by disabling hdfs, yarn, mapreduce, etc. modules.  This will reduce the development
time spend on compiling unrelated modules.
> Note that this is for local maven builds.   We are not proposing to change Jenkins builds,
which always build all the modules.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message