spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Sabater (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-8616) SQLContext doesn't handle tricky column names when loading from JDBC
Date Thu, 25 Jun 2015 22:17:05 GMT

    [ https://issues.apache.org/jira/browse/SPARK-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602051#comment-14602051
] 

David Sabater commented on SPARK-8616:
--------------------------------------

It looks the issue only appears on 1.3.2, I tested with 1.4.0 and using a JSON or CSV you
get error directly:
scala> df.save("spark/people.parquet")
warning: there were 1 deprecation warning(s); re-run with -deprecation for details
15/06/25 23:10:43 INFO MemoryStore: ensureFreeSpace(230688) called with curMem=271386, maxMem=278302556
15/06/25 23:10:43 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated
size 225.3 KB, free 264.9 MB)
15/06/25 23:10:43 INFO MemoryStore: ensureFreeSpace(19750) called with curMem=502074, maxMem=278302556
15/06/25 23:10:43 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated
size 19.3 KB, free 264.9 MB)
15/06/25 23:10:43 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on localhost:52751
(size: 19.3 KB, free: 265.4 MB)
15/06/25 23:10:43 INFO SparkContext: Created broadcast 5 from textFile at CsvRelation.scala:57
15/06/25 23:10:43 ERROR InsertIntoHadoopFsRelation: Aborting job.
java.lang.RuntimeException: Attribute name " age" contains invalid character(s) among " ,;{}()
	=". Please use alias to rename it.            
	at scala.sys.package$.error(package.scala:27)


> SQLContext doesn't handle tricky column names when loading from JDBC
> --------------------------------------------------------------------
>
>                 Key: SPARK-8616
>                 URL: https://issues.apache.org/jira/browse/SPARK-8616
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.4.0
>         Environment: Ubuntu 14.04, Sqlite 3.8.7, Spark 1.4.0
>            Reporter: Gergely Svigruha
>
> Reproduce:
>  - create a table in a relational database (in my case sqlite) with a column name containing
a space:
>  CREATE TABLE my_table (id INTEGER, "tricky column" TEXT);
>  - try to create a DataFrame using that table:
> sqlContext.read.format("jdbc").options(Map(
>   "url" -> "jdbs:sqlite:...",
>   "dbtable" -> "my_table")).load()
> java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (no such column:
tricky)
> According to the SQL spec this should be valid:
> http://savage.net.au/SQL/sql-99.bnf.html#delimited%20identifier



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message