hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sachin Jain <sachinjain...@gmail.com>
Subject Re: Issues with Spark On Hbase Connector
Date Mon, 29 Aug 2016 06:03:33 GMT
Hi Sudhir,

There is connection leak problem with hortonworks hbase connector if you
use hbase 1.2.0.
I tried to use hortonwork's connector and felt into the same problem.

Have a look at this Hbase issue HBASE-16017 [0]. The fix for this was
backported to 1.3.0, 1.4.0 and 2.0.0
I have raised a ticket on their github repo [1] and also generated PR to
get this fixed. Check this out [2].

But unfortunately no one has responded to it yet so it is not merged. But
this fix works and I am currently using the same without problems.
So if you want to use this. You can use this one [3] where changes for pull
request already there.

Hope it helps!!

[0]: https://issues.apache.org/jira/browse/HBASE-16017
[1]: https://github.com/hortonworks-spark/shc/issues/19
[2]: https://github.com/hortonworks-spark/shc/pull/20
[3]: https://github.com/sachinjain024/shc/tree/Issue-19-Connection-Leak


On Mon, Aug 29, 2016 at 7:31 AM, sudhir patil <spatil.sudhir@gmail.com>
wrote:

> Ok, thanks for the link Ted
>
> On Aug 29, 2016 9:54 AM, "Ted Yu" <yuzhihong@gmail.com> wrote:
>
> > For hortonworks product(s), consider raising question on
> > https://community.hortonworks.com
> >
> > FYI
> >
> > On Sun, Aug 28, 2016 at 6:45 PM, spats <spatil.sudhir@gmail.com> wrote:
> >
> > > Regarding hbase connector by hortonworks
> > > https://github.com/hortonworks-spark/shc, it would be great if someone
> > can
> > > answer these
> > >
> > > 1. What versions of Hbase & Spark expected? I could not run examples
> > > provided using spark 1.6.0 & hbase 1.2.0
> > > 2. I get error when i run example provided here , any pointers on what
> i
> > am
> > > doing wrong?
> > >
> > > looks like spark not reading hbase-site.xml, but passed it in --files
> > while
> > > spark-shell
> > > e.g --files
> > > /etc/hbase/conf/hbase-site.xml,/etc/hbase/conf/hdfs-site.
> > > xml,/etc/hbase/conf/core-site.xml
> > >
> > > error
> > > 16/08/27 12:35:00 WARN zookeeper.ClientCnxn: Session 0x0 for server
> null,
> > > unexpected error, closing socket connection and attempting reconnect
> > > java.net.ConnectException: Connection refused
> > >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > >         at
> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
> > >         at
> > > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(
> > > ClientCnxnSocketNIO.java:350)
> > >         at
> > > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
> > >
> > >
> > >
> > > --
> > > View this message in context: http://apache-hbase.679495.n3.
> > > nabble.com/Issues-with-Spark-On-Hbase-Connector-tp4082151.html
> > > Sent from the HBase User mailing list archive at Nabble.com.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message