hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abel Fernández <mevsmys...@gmail.com>
Subject hbase/spark - Delegation Token can be issued only with kerberos or web authentication
Date Fri, 18 Nov 2016 12:01:00 GMT
Hello,

We are having problems with the delegation of the token in a secure
cluster: Delegation Token can be issued only with kerberos or web
authentication

We have a spark process which is generating the hfiles to be loaded into
hbase. To generate these hfiles, (we are using a back-ported version of the
latest hbase/spark code), we are using this method
HBaseRDDFunctions.hbaseBulkLoadThinRows.

I think the problem is in the below piece of code. This function is
executed in every partition of the rdd, when the executors are trying to
execute the code, the executors do not have a valid kerberos credential and
cannot execute anything.

private def hbaseForeachPartition[T](configBroadcast:

 Broadcast[SerializableWritable[Configuration]],
                                        it: Iterator[T],
                                        f: (Iterator[T], Connection) =>
Unit) = {

    val config = getConf(configBroadcast)

    applyCreds
    // specify that this is a proxy user
    val smartConn = HBaseConnectionCache.getConnection(config)
    f(it, smartConn.connection)
    smartConn.close()
  }

I have attached the spark-submit and the complete error log trace. Has
anyone faced this problem before?

Thanks in advance.

Regards,
Abel.
-- 
Un saludo - Best Regards.
Abel

Mime
View raw message