spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From steveloughran <>
Subject [GitHub] spark pull request #12004: [SPARK-7481] [build] Add spark-cloud module to pu...
Date Wed, 23 Nov 2016 13:31:49 GMT
Github user steveloughran commented on a diff in the pull request:
    --- Diff: pom.xml ---
    @@ -2558,6 +2660,26 @@
    +    <!--
    +      The cloud profile enables the cloud module.
    +      It does not declare the hadoop-* artifacts which
    +      the cloud module pulls in; these are delegated to
    +      the hadoop-x.y protocols, so permitting different
    +      hadoop versions to declare different include/exclude
    +      rules (especially transient dependencies).
    +      To use this profile, the hadoop-2.7 profile must also
    --- End diff --
    you'd never want to have cloud without hadoop-2.7, but you may want to do hadoop-2.7 without
cloud. That really mattered on spark-1.6, as it would make for a very large spark-assembly;
in 2.x it will result in more files in SPARK_HOME/jars, and for a bigger spark tarball.
    I'd left it as an option, as with hive, mesos and yarn. However, if you do try to build
without hadoop-2.7 set, things won't build

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message