drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@dremio.com>
Subject Re: Increased build time and failure?
Date Tue, 27 Oct 2015 00:52:57 GMT
Some things to note:

FMPP does an incremental build now (not replacing the generated files if
they haven't changed, thus meaning javac also avoids compilation). This
makes an incremental build way faster. (The enhancement also avoids double
copying/compiling the generated classes.) Since you did clean, you won't
see that improvement.

The bigger win on build time is with tests running: startup and shutdown of
Drillbits is faster (due to classpath scanning savings, function
implementation registry creation and reduced shutdown delays in netty).

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Mon, Oct 26, 2015 at 5:45 PM, Sudheesh Katkam <skatkam@maprtech.com>
wrote:

> I added MAVEN_OPTS and now the build time completes successfully in ~7
> minutes on multiple runs. The commit message says “reduces build time by
> 30%”, but I do not see any improvement. What am I doing wrong?
>
> - Sudheesh
>
> > On Oct 26, 2015, at 2:51 PM, Jacques Nadeau <jacques@dremio.com> wrote:
> >
> > Have you seen this multiple times? The first time you may build with the
> > changes I merged today include downloading a new hadoop version (which
> > takes forever the first time).
> >
> > In general, the build without tests is substantially faster than before.
> I
> > have seen PermGen issues and have set my environment variables to provide
> > more permgen. I use MAVEN_OPTS="-Xmx1g -XX:MaxPermSize=256m".
> >
> >
> >
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Mon, Oct 26, 2015 at 2:34 PM, Sudheesh Katkam <skatkam@maprtech.com>
> > wrote:
> >
> >> Hey y’all,
> >>
> >> I use “mvn clean install -DskipTests” to build Drill on my Mac. On the
> >> latest master, I am getting this error now. Also notice that the build
> time
> >> is ~23 minutes; it used to be ~7 minutes.
> >>
> >> @Julien what are the implications of your changes in DRILL-3742?
> >>
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] Reactor Summary:
> >> [INFO]
> >> [INFO] Apache Drill Root POM .............................. SUCCESS
> [02:29
> >> min]
> >> [INFO] tools/Parent Pom ................................... SUCCESS [
> >> 23.473 s]
> >> [INFO] tools/freemarker codegen tooling ................... SUCCESS
> [01:41
> >> min]
> >> [INFO] Drill Protocol ..................................... SUCCESS [
> >> 34.936 s]
> >> [INFO] Common (Logical Plan, Base expressions) ............ SUCCESS
> [01:03
> >> min]
> >> [INFO] contrib/Parent Pom ................................. SUCCESS [
> >> 3.533 s]
> >> [INFO] contrib/data/Parent Pom ............................ SUCCESS [
> >> 10.862 s]
> >> [INFO] contrib/data/tpch-sample-data ...................... SUCCESS [
> >> 17.306 s]
> >> [INFO] exec/Parent Pom .................................... SUCCESS [
> >> 3.997 s]
> >> [INFO] exec/Java Execution Engine ......................... SUCCESS
> [08:16
> >> min]
> >> [INFO] exec/JDBC Driver using dependencies ................ SUCCESS [
> >> 52.674 s]
> >> [INFO] JDBC JAR with all dependencies ..................... SUCCESS
> [01:17
> >> min]
> >> [INFO] contrib/mongo-storage-plugin ....................... SUCCESS [
> >> 53.952 s]
> >> [INFO] contrib/hbase-storage-plugin ....................... SUCCESS [
> >> 51.397 s]
> >> [INFO] contrib/jdbc-storage-plugin ........................ SUCCESS [
> >> 14.904 s]
> >> [INFO] contrib/hive-storage-plugin/Parent Pom ............. SUCCESS [
> >> 1.964 s]
> >> [INFO] contrib/hive-storage-plugin/hive-exec-shaded ....... SUCCESS [
> >> 41.461 s]
> >> [INFO] contrib/hive-storage-plugin/core ................... SUCCESS
> [01:34
> >> min]
> >> [INFO] contrib/drill-gis-plugin ........................... SUCCESS [
> >> 41.361 s]
> >> [INFO] Packaging and Distribution Assembly ................ SKIPPED
> >> [INFO] contrib/sqlline .................................... SKIPPED
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] BUILD SUCCESS
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] Total time: 22:51 min
> >> [INFO] Finished at: 2015-10-26T14:22:50-07:00
> >> [INFO] Final Memory: 139M/1714M
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [ERROR] PermGen space -> [Help 1]
> >> [ERROR]
> >> [ERROR] To see the full stack trace of the errors, re-run Maven with the
> >> -e switch.
> >> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> >> [ERROR]
> >> [ERROR] For more information about the errors and possible solutions,
> >> please read the following articles:
> >> [ERROR] [Help 1]
> >> http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
> >>
> >> Thank you,
> >> Sudheesh
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message