spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prabhu Joseph (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-16225) Spark Sql throws ArrayIndexOutOfBoundsException on accessing Null Value
Date Mon, 27 Jun 2016 09:47:52 GMT

    [ https://issues.apache.org/jira/browse/SPARK-16225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350718#comment-15350718
] 

Prabhu Joseph commented on SPARK-16225:
---------------------------------------

Thanks [~srowen]

> Spark Sql throws ArrayIndexOutOfBoundsException on accessing Null Value
> -----------------------------------------------------------------------
>
>                 Key: SPARK-16225
>                 URL: https://issues.apache.org/jira/browse/SPARK-16225
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.5.2
>            Reporter: Prabhu Joseph
>
> Spark Sql throws ArrayIndexOutOfBoundsException on accessing Null value
> Repro:
> {code}
> [root@spark1 spark]# cat repro 
> a,b,c,,,
> ./bin/spark-shell
> scala> val pTel = sc.textFile("file:///usr/hdp/2.3.4.7-4/spark/repro")
> scala> pTel.take(1)
> res0: Array[String] = Array(a,b,c,,,)
> scala> val delimiter = ","
> delimiter: String = ,
> scala> import org.apache.spark.sql._
> import org.apache.spark.sql._
> scala> val pRowRDD = pTel.map(_.split(delimiter)).map(p => Row(p(0),p(1),p(2),p(3),p(4),p(5)))
> pRowRDD: org.apache.spark.rdd.RDD[org.apache.spark.sql.Row] = MapPartitionsRDD[3] at
map at <console>:28
> scala> pRowRDD.take(1)
> 16/06/27 08:21:40 ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 1)
> java.lang.ArrayIndexOutOfBoundsException: 3
> {code}
> It works fine when the data is a,b,c,d,e,f



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message