Mailing-List: contact user-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@ignite.apache.org
Date: Thu, 6 Oct 2016 07:56:56 -0700 (MST)
From: Tracyl <tliang36@bloomberg.net>
To: user@ignite.apache.org
Message-ID: <1475765816436-8117.post@n6.nabble.com>
Subject: Does IgniteCache could be access thr Spark JDBC data source api?
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
archived-at: Thu, 06 Oct 2016 14:57:01 -0000

Hey team,

I was able to use JDBC driver tool to access IgniteCache. Is it possible to
connect to Ignite thr Spark JDBC data source api? Below are the code and
exceptions I got. It seems like the connection is successful but there are
datatype mapping issues. Do I need to define some schema from the server
side? Any suggestion would be really helpful for me. Thanks in advance! 

Sample Code:
val props = Map(
      "url" -> "jdbc:ignite://ip:11211/TestCache",
      "driver" -> "org.apache.ignite.IgniteJdbcDriver",
      "dbtable" -> "Person"
    ) 
val personDF = ctx.read.format("jdbc").options(props).load()

Exceptions:
Exception in thread "main" java.sql.SQLException: Unsupported type 1111
        at
org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.org$apache$spark$sql$execution$datasources$jdbc$JDBCRDD$$getCatalystType(JDBCRDD.scala:103)
        at
org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$$anonfun$1.apply(JDBCRDD.scala:140)
        at
org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$$anonfun$1.apply(JDBCRDD.scala:140)
        at scala.Option.getOrElse(Option.scala:120)
        at
org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:139)
        at
org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation.<init>(JDBCRelation.scala:91)
        at
org.apache.spark.sql.execution.datasources.jdbc.DefaultSource.createRelation(DefaultSource.scala:60)
        at
org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:158)
        at
org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:119)
        at
ignite.spark.DataFrameLoader$.personTest(DataFrameLoader.scala:131)
        at ignite.spark.DataFrameLoader.personTest(DataFrameLoader.scala)
        at jdbc.PersonLoader.executeTransaction(PersonLoader.java:175)
        at jdbc.PersonLoader.main(PersonLoader.java:115)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
        at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
        at
org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)


--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Does-IgniteCache-could-be-access-thr-Spark-JDBC-data-source-api-tp8117.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.