hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerry He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
Date Mon, 13 Mar 2017 05:30:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15906895#comment-15906895

Jerry He commented on HBASE-16179:

Hi, Ted

Sorry for the delay in review.  I think it overall looks good!  Most of the things are in

A few comments.

[~busbey] mentioned about the assembly tarball in his previous comment.  If my understanding
is correct, each assembly tarball can only include one version the hbase spark jar.
I didn't see how we accomplish it.

Also an overall comment on the supported combinations. We have four and we are building and
possibly deploying to maven repo all of these four.  My thinking is that it seems to be an
overkill, We are a HBase project, not a Spark or Scala project.  What about we have two compat
module: one for spark 1.6 and one for spark 2?  Spark 2 defaults with Scala 2.11 out of box.
 Spark 1.6 defaults with Scala 2.10 out of box.  

We claim support for the other two combinations.  But if needed, user needs to compile from
source with -Dscala.version=xxx to get the one that is not available out of the box.
That will simply our build and distribution.
My 2 cents.

> Fix compilation errors when building hbase-spark against Spark 2.0
> ------------------------------------------------------------------
>                 Key: HBASE-16179
>                 URL: https://issues.apache.org/jira/browse/HBASE-16179
>             Project: HBase
>          Issue Type: Bug
>          Components: spark
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>            Priority: Critical
>             Fix For: 2.0.0
>         Attachments: 16179.v0.txt, 16179.v10.txt, 16179.v11.txt, 16179.v12.txt, 16179.v12.txt,
16179.v12.txt, 16179.v13.txt, 16179.v15.txt, 16179.v16.txt, 16179.v18.txt, 16179.v19.txt,
16179.v19.txt, 16179.v1.txt, 16179.v1.txt, 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v4.txt,
16179.v5.txt, 16179.v7.txt, 16179.v8.txt, 16179.v9.txt
> I tried building hbase-spark module against Spark-2.0 snapshot and got the following
compilation errors:
> http://pastebin.com/bg3w247a
> Some Spark classes such as DataTypeParser and Logging are no longer accessible to downstream
> hbase-spark module should not depend on such classes.

This message was sent by Atlassian JIRA

View raw message