hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Malaska (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15225) Connecting to HBase via newAPIHadoopRDD in PySpark gives org.apache.hadoop.hbase.client.RetriesExhaustedException
Date Sat, 06 Feb 2016 21:26:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136013#comment-15136013
] 

Ted Malaska commented on HBASE-15225:
-------------------------------------

OK I have updated the Github.  This has everything that is in HBase but with the scan kerberos
fix plus it is back ported to CDH 5.5.  If you are not using CDH then just make the needed
changes and fork the code.

https://github.com/tmalaska/SparkOnHBase

This week I will work with the HBase committers to get the kerberos fit checked in.

Let me know if there is anything else you need.

> Connecting to HBase via newAPIHadoopRDD in PySpark gives  org.apache.hadoop.hbase.client.RetriesExhaustedException
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-15225
>                 URL: https://issues.apache.org/jira/browse/HBASE-15225
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.98.4
>         Environment: spark 1.6.0 , Hbase 0.98.4, kerberos,  hbase.rpc.protection set
to authentication.
>            Reporter: Sanjay Kumar
>
> Unable to read HBase table into Spark with hbase security authentication set to kerberos.
Seeing the following error. 
> : org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=31,
exceptions:
> Thu Feb 04 22:01:55 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:01:56 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:01:56 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:01:57 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:01:59 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:02:03 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:02:13 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:02:23 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:02:34 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:02:44 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:03:04 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:03:24 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:03:44 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:04:04 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:04:24 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:04:44 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> Thu Feb 04 22:05:04 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.IOException: Connection reset by peer
> .
> .
> .
> Thu Feb 04 22:09:46 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
java.io.EOFException
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:129)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
> at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:282)
> at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:187)
> at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:182)
> at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:109)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:738)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:178)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:82)
> at org.apache.hadoop.hbase.client.MetaScanner.allTableRegions(MetaScanner.java:282)
> at org.apache.hadoop.hbase.client.HTable.getRegionLocations(HTable.java:616)
> at org.apache.hadoop.hbase.util.RegionSizeCalculator.<init>(RegionSizeCalculator.java:79)
> at org.apache.hadoop.hbase.util.RegionSizeCalculator.<init>(RegionSizeCalculator.java:64)
> at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:160)
> at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:115)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
> at scala.Option.getOrElse(Option.scala:120)
> at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
> at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
> at scala.Option.getOrElse(Option.scala:120)
> at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
> at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1277)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
> at org.apache.spark.rdd.RDD.take(RDD.scala:1272)
> at org.apache.spark.api.python.SerDeUtil$.pairRDDToPython(SerDeUtil.scala:202)
> at org.apache.spark.api.python.PythonRDD$.newAPIHadoopRDD(PythonRDD.scala:530)
> at org.apache.spark.api.python.PythonRDD.newAPIHadoopRDD(PythonRDD.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:207)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
on local exception: java.io.EOFException
> at org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1484)
> at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1456)
> at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1656)
> at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1714)
> at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:29900)
> at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:308)
> at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:164)
> at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:59)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)
> ... 42 more
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:392)
> at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1071)
> at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message