hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
Date Wed, 18 Jan 2017 22:41:26 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15828901#comment-15828901
] 

Sean Busbey commented on HBASE-16179:
-------------------------------------

* please things flagged by the qabot (e.g. whitespace issues)
* we're adding ~7 modules here. could we organize them under a single parent module for hbase-spark?
* We shouldn't have pom entries like this, because they have led to us mistakenly skipping
tests before:
{code}
+            <plugin>
+                <artifactId>maven-surefire-plugin</artifactId>
+                <!-- Always skip the second part executions, since we only run
+          simple unit tests in this module -->
+                <executions>
+                    <execution>
+                        <id>secondPartTestsExecution</id>
+                        <phase>test</phase>
+                        <goals>
+                            <goal>test</goal>
+                        </goals>
+                        <configuration>
+                            <skip>true</skip>
+                        </configuration>
+                    </execution>
+                </executions>
+            </plugin>
{code}
* Please add docs to the ref guide explaining how to make use of these artifacts in a downstream
project
* Could you add comments explaining the use of this profile? I thought we were building all
spark x scala versions (and are AFAICT)?
{code}
diff --git a/hbase-spark/pom.xml b/hbase-spark/pom.xml
index 035dfcc..e195b9c 100644
--- a/hbase-spark/pom.xml
+++ b/hbase-spark/pom.xml
@@ -686,6 +685,24 @@
     </build>
 
     <profiles>
+    <profile>
+      <id>spark-2.0</id>
+      <activation>
+        <property>
+            <!--h2--><name>!spark.profile</name>
+        </property>
+      </activation>
+      <properties>
+        <spark.version>2.0.2</spark.version>
+      </properties>
+      <dependencies>
+        <dependency>
+            <groupId>org.apache.hbase</groupId>
+            <artifactId>hbase-spark2.0-compat</artifactId>
+            <version>${project.version}</version>
+        </dependency>
+      </dependencies>
+    </profile>
{code}
* Where in the assembly do the generated artifacts show up?

> Fix compilation errors when building hbase-spark against Spark 2.0
> ------------------------------------------------------------------
>
>                 Key: HBASE-16179
>                 URL: https://issues.apache.org/jira/browse/HBASE-16179
>             Project: HBase
>          Issue Type: Bug
>          Components: spark
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>            Priority: Critical
>             Fix For: 2.0.0
>
>         Attachments: 16179.v0.txt, 16179.v10.txt, 16179.v11.txt, 16179.v12.txt, 16179.v12.txt,
16179.v12.txt, 16179.v13.txt, 16179.v15.txt, 16179.v16.txt, 16179.v18.txt, 16179.v19.txt,
16179.v19.txt, 16179.v1.txt, 16179.v1.txt, 16179.v4.txt, 16179.v5.txt, 16179.v7.txt, 16179.v8.txt,
16179.v9.txt
>
>
> I tried building hbase-spark module against Spark-2.0 snapshot and got the following
compilation errors:
> http://pastebin.com/bg3w247a
> Some Spark classes such as DataTypeParser and Logging are no longer accessible to downstream
projects.
> hbase-spark module should not depend on such classes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message