hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ari Rabkin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15184) SparkSQL Scan operation doesn't work on kerberos cluster
Date Tue, 23 Feb 2016 10:14:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15158661#comment-15158661
] 

Ari Rabkin commented on HBASE-15184:
------------------------------------

Thanks so much for tackling this, [~ted.m].  Is there some way I could put in a no-op spark
Map early in the job to work around this?
The core bit of code in my app looks like this:

    val rawTable = sc.newAPIHadoopRDD(hbconf, classOf[TableInputFormat],
      classOf[org.apache.hadoop.hbase.io.ImmutableBytesWritable],
      classOf[org.apache.hadoop.hbase.client.Result])
    val resultsDataset = rawTable.map( resultToRecord)


Note that there's no explicit scan happening here; it's all inside the TableInputFormat. So
I'm a bit unclear what I would do to cause a 'map' operation to happen early enough. 

> SparkSQL Scan operation doesn't work on kerberos cluster
> --------------------------------------------------------
>
>                 Key: HBASE-15184
>                 URL: https://issues.apache.org/jira/browse/HBASE-15184
>             Project: HBase
>          Issue Type: Bug
>          Components: spark
>            Reporter: Ted Malaska
>            Assignee: Ted Malaska
>            Priority: Critical
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15184.1.patch, HBaseSparkModule.zip
>
>
> I was using the HBase Spark Module at a client with Kerberos and I ran into an issue
with the Scan.  
> I made a fix for the client but we need to put it back into HBase.  I will attach my
solution, but it has a major problem.  I had to over ride a protected class in spark.  I will
need help to decover a better approach



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message