spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sro...@apache.org
Subject spark git commit: [SPARK-7669] Builds against Hadoop 2.6+ get inconsistent curator depend…
Date Sun, 17 May 2015 16:03:25 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.4 898be6248 -> 0feb3ded2


[SPARK-7669] Builds against Hadoop 2.6+ get inconsistent curator depend…

This adds a new profile, `hadoop-2.6`, copying over the hadoop-2.4 properties, updating ZK
to 3.4.6 and making the curator version a configurable option. That keeps the curator-recipes
JAR in sync with that used in hadoop.

There's one more option to consider: making the full curator-client version explicit with
its own dependency version. This will pin down the version from hadoop and hive imports

Author: Steve Loughran <stevel@hortonworks.com>

Closes #6191 from steveloughran/stevel/SPARK-7669-hadoop-2.6 and squashes the following commits:

e3e281a [Steve Loughran] SPARK-7669 declare the version of curator-client and curator-framework
JARs
2901ea9 [Steve Loughran] SPARK-7669 Builds against Hadoop 2.6+ get inconsistent curator dependencies

(cherry picked from commit 50217667cc1239ed3b15f4d10907b727ed85d7fa)
Signed-off-by: Sean Owen <sowen@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0feb3ded
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/0feb3ded
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/0feb3ded

Branch: refs/heads/branch-1.4
Commit: 0feb3ded2e1681e0c282b73a1ec5af454bc78a13
Parents: 898be62
Author: Steve Loughran <stevel@hortonworks.com>
Authored: Sun May 17 17:03:11 2015 +0100
Committer: Sean Owen <sowen@cloudera.com>
Committed: Sun May 17 17:03:20 2015 +0100

----------------------------------------------------------------------
 pom.xml | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/0feb3ded/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 1b45cdb..6768a03 100644
--- a/pom.xml
+++ b/pom.xml
@@ -130,6 +130,7 @@
     <hbase.artifact>hbase</hbase.artifact>
     <flume.version>1.4.0</flume.version>
     <zookeeper.version>3.4.5</zookeeper.version>
+    <curator.version>2.4.0</curator.version>
     <hive.group>org.spark-project.hive</hive.group>
     <!-- Version used in Maven Hive dependency -->
     <hive.version>0.13.1a</hive.version>
@@ -707,7 +708,7 @@
       <dependency>
         <groupId>org.apache.curator</groupId>
         <artifactId>curator-recipes</artifactId>
-        <version>2.4.0</version>
+        <version>${curator.version}</version>
         <scope>${hadoop.deps.scope}</scope>
         <exclusions>
           <exclusion>
@@ -717,6 +718,16 @@
         </exclusions>
       </dependency>
       <dependency>
+        <groupId>org.apache.curator</groupId>
+        <artifactId>curator-client</artifactId>
+        <version>${curator.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.curator</groupId>
+        <artifactId>curator-framework</artifactId>
+        <version>${curator.version}</version>
+      </dependency>
+      <dependency>
         <groupId>org.apache.hadoop</groupId>
         <artifactId>hadoop-client</artifactId>
         <version>${hadoop.version}</version>
@@ -1680,6 +1691,17 @@
     </profile>
 
     <profile>
+      <id>hadoop-2.6</id>
+      <properties>
+        <hadoop.version>2.6.0</hadoop.version>
+        <jets3t.version>0.9.3</jets3t.version>
+        <commons.math3.version>3.1.1</commons.math3.version>
+        <zookeeper.version>3.4.6</zookeeper.version>
+        <curator.version>2.6.0</curator.version>
+      </properties>
+    </profile>
+
+    <profile>
       <id>yarn</id>
       <modules>
         <module>yarn</module>
@@ -1709,7 +1731,7 @@
         <dependency>
           <groupId>org.apache.curator</groupId>
           <artifactId>curator-recipes</artifactId>
-          <version>2.4.0</version>
+          <version>${curator.version}</version>
           <exclusions>
             <exclusion>
               <groupId>org.apache.zookeeper</groupId>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message