hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zoltan Haindrich (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-14735) Build Infra: Spark artifacts download takes a long time
Date Fri, 28 Oct 2016 20:28:58 GMT

     [ https://issues.apache.org/jira/browse/HIVE-14735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zoltan Haindrich updated HIVE-14735:
------------------------------------
    Attachment: HIVE-14735.1.patch

[~spena] it's good to know these things...i've tryed this out - because in gradle this would
be easy...i assumed maven can do it too...well, it seems it does! but it needs quite a bunch
of xml to do something like this ;)

i've experimented with it...and it looks like it works - i would like to submit a test ptest
run to check that everything is all right - but since my own server serves this spark related
maven repo, I don't really want it to go in ;)

i've published a preliminary "conceptional" repackaging tool here:
https://github.com/kgyrtkirk/hive-14735

[~spena], you might want to try it out locally, i've sketched a readme in that git repo -
hope it helps evaluating this maven option.

I think this will eventually work...download/unpack/etc is done by maven plugins which should
be highly portable.

notes:

* {{mvn clean}} clears the unpacked things - which is good
* unpacking a new version doesn't remove the old files, just pastes the new tree on top of
it...but in case someone changes distinct branches I think he will use {{mvn clean}} or a
harder {{git clean -dfx}} - so it should be ok
* there is a log4j2 properties filewhich gets copied into this unpacked directory...it can
be included in the artifact...or keep it like this?


> Build Infra: Spark artifacts download takes a long time
> -------------------------------------------------------
>
>                 Key: HIVE-14735
>                 URL: https://issues.apache.org/jira/browse/HIVE-14735
>             Project: Hive
>          Issue Type: Bug
>          Components: Build Infrastructure
>            Reporter: Vaibhav Gumashta
>            Assignee: Zoltan Haindrich
>         Attachments: HIVE-14735.1.patch
>
>
> In particular this command:
> {{curl -Sso ./../thirdparty/spark-1.6.0-bin-hadoop2-without-hive.tgz http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.6.0-bin-hadoop2-without-hive.tgz}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message