hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17822) Provide an option to skip shading of jars
Date Mon, 16 Oct 2017 22:53:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16206727#comment-16206727
] 

Prasanth Jayachandran commented on HIVE-17822:
----------------------------------------------

Some numbers
{code}
# Base: Clean offline quite build
$ time mvn clean install -DskipTests -o -q
real	3m9.005s
user	7m14.864s
sys	0m40.295s

# Parallel (using 1C gave best build times) build
$ time mvn clean install -DskipTests -T 1C -o -q
real	2m24.415s
user	8m12.243s
sys	0m54.905s

# With MAVEN_OPTS
$ time MAVEN_OPTS="-XX:+TieredCompilation -XX:TieredStopAtLevel=1" mvn clean install -DskipTests
-T 1C -o -q
real	2m12.872s
user	7m46.879s
sys	0m49.696s

# Skip clean
$ MAVEN_OPTS="-XX:+TieredCompilation -XX:TieredStopAtLevel=1"
$ time MAVEN_OPTS="-XX:+TieredCompilation -XX:TieredStopAtLevel=1" mvn install -DskipTests
-T 1C -o -q
real	1m31.403s
user	5m13.439s
sys	0m37.885s

# Skip shade for jdbc and druid-handler (requires HIVE-17822)
# NOTE: if you are changing/testing jdbc or druid you may want to skip this step
$ time MAVEN_OPTS="-XX:+TieredCompilation -XX:TieredStopAtLevel=1" mvn install -DskipShade
-DskipTests -T 1C -o -q
real	1m20.130s
user	4m37.645s
sys	0m39.897s

# Skip remote resource plugin
$ time MAVEN_OPTS="-XX:+TieredCompilation -XX:TieredStopAtLevel=1" mvn install -DskipShade
-DskipTests -Dremoteresources.skip=true -T 1C -o -q
real	0m37.485s
user	0m52.652s
sys	0m14.118s

# Build ql and downstream modules
$ time MAVEN_OPTS="-XX:+TieredCompilation -XX:TieredStopAtLevel=1" mvn install -DskipShade
-DskipTests -Dremoteresources.skip=true -T 1C -o -q -pl ql -amd
real	0m31.827s
user	1m50.349s
sys	0m9.494s

# Build llap-server and downstream modules
$ time MAVEN_OPTS="-XX:+TieredCompilation -XX:TieredStopAtLevel=1" mvn install -DskipShade
-DskipTests -Dremoteresources.skip=true -T 1C -o -q -pl llap-server -amd
real	0m9.147s
user	0m20.189s
sys	0m3.056s
{code}

> Provide an option to skip shading of jars
> -----------------------------------------
>
>                 Key: HIVE-17822
>                 URL: https://issues.apache.org/jira/browse/HIVE-17822
>             Project: Hive
>          Issue Type: Bug
>          Components: Build Infrastructure
>    Affects Versions: 3.0.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>         Attachments: HIVE-17822.1.patch
>
>
> Maven shade plugin does not have option to skip. Adding it under a profile can help with
skip shade reducing build times.
> Maven build profile shows druid and jdbc shade plugin to be slowest (also hive-exec).
For devs not working on druid or jdbc, it will be good to have an option to skip shading via
a profile. With this it will be possible to get a subsecond dev build.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message