hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
Date Mon, 01 May 2017 21:19:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15991564#comment-15991564

Yi Liang commented on HBASE-16179:

   I just review the patch, and have some thoughts on this jira,  I think 6 hbase-sparkXXX
folders is too many, and how about the structure as below, it is more clear and simple
       - hbase-spark-1.6 (diff code for spark1.6)
       - hbase-spark-2.0 (diff code for spark2.0)
       - src (common code)
       - pom.xml
We still have a hbase-spark as parent structure, and add two sub-folder, one is for spark2.0
and one is for spark1.6; and I also check the code, for both scala 2.10 and 2.11, there are
no code difference between the two sub-folder, we can make hbase-spark-1.6 default support
scala 2.10 and hbase-spark-2.0 default support scala 2.11, and if user want to change scala
version, they can modify pom file under those two sub folder or use some mvn option like -Psparkscalaversion
= xxx.xxx  

> Fix compilation errors when building hbase-spark against Spark 2.0
> ------------------------------------------------------------------
>                 Key: HBASE-16179
>                 URL: https://issues.apache.org/jira/browse/HBASE-16179
>             Project: HBase
>          Issue Type: Bug
>          Components: spark
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>            Priority: Critical
>              Labels: build
>             Fix For: 2.0.0
>         Attachments: 16179.v0.txt, 16179.v10.txt, 16179.v11.txt, 16179.v12.txt, 16179.v12.txt,
16179.v12.txt, 16179.v13.txt, 16179.v15.txt, 16179.v16.txt, 16179.v18.txt, 16179.v19.txt,
16179.v19.txt, 16179.v1.txt, 16179.v1.txt, 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt,
16179.v25.txt, 16179.v26.txt, 16179.v27.txt, 16179.v4.txt, 16179.v5.txt, 16179.v7.txt, 16179.v8.txt,
> I tried building hbase-spark module against Spark-2.0 snapshot and got the following
compilation errors:
> http://pastebin.com/bg3w247a
> Some Spark classes such as DataTypeParser and Logging are no longer accessible to downstream
> hbase-spark module should not depend on such classes.

This message was sent by Atlassian JIRA

View raw message