hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zoltan Haindrich (JIRA)" <>
Subject [jira] [Updated] (HIVE-14735) Build Infra: Spark artifacts download takes a long time
Date Fri, 28 Oct 2016 20:28:58 GMT


Zoltan Haindrich updated HIVE-14735:
    Attachment: HIVE-14735.1.patch

[~spena] it's good to know these things...i've tryed this out - because in gradle this would
be easy...i assumed maven can do it too...well, it seems it does! but it needs quite a bunch
of xml to do something like this ;)

i've experimented with it...and it looks like it works - i would like to submit a test ptest
run to check that everything is all right - but since my own server serves this spark related
maven repo, I don't really want it to go in ;)

i've published a preliminary "conceptional" repackaging tool here:

[~spena], you might want to try it out locally, i've sketched a readme in that git repo -
hope it helps evaluating this maven option.

I think this will eventually is done by maven plugins which should
be highly portable.


* {{mvn clean}} clears the unpacked things - which is good
* unpacking a new version doesn't remove the old files, just pastes the new tree on top of
it...but in case someone changes distinct branches I think he will use {{mvn clean}} or a
harder {{git clean -dfx}} - so it should be ok
* there is a log4j2 properties filewhich gets copied into this unpacked can
be included in the artifact...or keep it like this?

> Build Infra: Spark artifacts download takes a long time
> -------------------------------------------------------
>                 Key: HIVE-14735
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Build Infrastructure
>            Reporter: Vaibhav Gumashta
>            Assignee: Zoltan Haindrich
>         Attachments: HIVE-14735.1.patch
> In particular this command:
> {{curl -Sso ./../thirdparty/spark-1.6.0-bin-hadoop2-without-hive.tgz}}

This message was sent by Atlassian JIRA

View raw message