hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Wilson <eric.wil...@aver.io>
Subject Re: Writing to HBase from Spark
Date Tue, 18 Apr 2017 19:02:02 GMT
So if I'm using PySpark 2.1 and HBase 1.2.5 there is no way for them to
communicate with each other?

Eric

On Tue, Apr 18, 2017 at 2:03 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> There is some document here (slightly outdated) :
> http://hbase.apache.org/book.html#spark
>
> Currently hbase-spark module in master branch only supports Spark 1.6.x
>
> See HBASE-16179 for on-going work for Spark 2.y
>
> FYI
>
> On Tue, Apr 18, 2017 at 10:58 AM, Eric Wilson <eric.wilson@aver.io> wrote:
>
> > I've been spending a few days trying to figure out how to write to HBase
> > from a Spark RDD -- using Python.
> >
> > I recently found that Spark has removed some of it's pythonconverters,
> > because (as this commit
> > <https://github.com/apache/spark/commit/a680562a6f87a03a00f71bad1c4242
> > 67ae75c641>
> >  shows):
> > *"HBase has had Spark bindings for **a while and is even including them
> in
> > the HBase distribution in the next **version, making the examples
> obsolete.
> > The same applies to Cassandra, which **seems to have a proper Spark
> binding
> > library already."*
> >
> > Can anyone point me to documentation of Spark/HBase/Python integration?
> >
> > Thanks.
> >
> >
> > --
> >
> > <http://aver.io> <http://aver.io>[image: aver] <http://aver.io>
> > Eric Wilson
> > Software Engineer
> >
> > --
> >
> > ------------------------------
> > This email, including attachments, may contain information that is
> > privileged, confidential or is exempt from disclosure under applicable
> law
> > (including, but not limited to, protected health information). It is not
> > intended for transmission to, or receipt by, any unauthorized persons. If
> > the reader of this message is not the intended recipient, or the employee
> > or agent responsible for delivering the message to the intended
> recipient,
> > you are hereby notified that any dissemination, distribution or copying
> of
> > this communication is strictly prohibited. If you believe this email was
> > sent to you in error, do not read it. Please notify the sender
> immediately
> > informing them of the error and delete all copies and attachments of the
> > message from your system. Thank you.
> >
>



-- 


--

<http://aver.io>[image: aver] <http://aver.io> <http://aver.io>
Eric Wilson
Software Engineer

-- 

------------------------------
This email, including attachments, may contain information that is 
privileged, confidential or is exempt from disclosure under applicable law 
(including, but not limited to, protected health information). It is not 
intended for transmission to, or receipt by, any unauthorized persons. If 
the reader of this message is not the intended recipient, or the employee 
or agent responsible for delivering the message to the intended recipient, 
you are hereby notified that any dissemination, distribution or copying of 
this communication is strictly prohibited. If you believe this email was 
sent to you in error, do not read it. Please notify the sender immediately 
informing them of the error and delete all copies and attachments of the 
message from your system. Thank you.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message