spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juan Rodríguez Hortalá <juan.rodriguez.hort...@gmail.com>
Subject Re: how to implement my own datasource?
Date Thu, 25 Jun 2015 08:27:02 GMT
Hi,

You can connect to by JDBC as described in
https://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases.
Other option is using HadoopRDD and NewHadoopRDD to connect to databases
compatible with Hadoop, like HBase, some examples can be found at chapter 5
of "Learning Spark"
https://books.google.es/books?id=tOptBgAAQBAJ&pg=PT190&dq=learning+spark+hadooprdd&hl=en&sa=X&ei=4bqLVaDaLsXaU46NgfgL&ved=0CCoQ6AEwAA#v=onepage&q=%20hadooprdd&f=false
For Spark Streaming see the section "Custom Sources" of
https://spark.apache.org/docs/latest/streaming-programming-guide.html

Hope that helps.

Greetings,

Juan

2015-06-25 8:25 GMT+02:00 诺铁 <notyycn@gmail.com>:

> hi,
>
> I can't find documentation about datasource api,  how to implement custom
> datasource.  any hint is appreciated.    thanks.
>

Mime
View raw message