spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anbu <anbutec...@gmail.com>
Subject NumberFormatException while reading and split the file
Date Wed, 04 Apr 2018 08:13:18 GMT
1st Approach:

error :  value split is not a member of org.apache.spark.sql.Row?

val newRdd = spark.read.text("/xyz/a/b/filename").rdd

anotherRDD = newRdd.
    map(ip =>ip.split("\\|")).map(ip => Row(if (ip(0).isEmpty()) {
null.asInstanceOf[Int] }
                                                    else ip(0).toInt, ip(1),
ip(2), ip(3), ip(4), ip(5))
													
I'm getting the error in the  line 'ip.split("\\|")' value split is not a
member of org.apache.spark.sql.Row?
 
 
Another approach:
 
 error:"java.lang.NumberFormatException: For input string:
 
 
 val newRdd = spark.read.text("/xyz/a/b/filename").rdd

anotherRDD = newRdd.
    map(ip =>ip.toString().split("\\|")).map(ip => Row(if (ip(0).isEmpty())
{ null.asInstanceOf[Int] }
                                                    else ip(0).toInt, ip(1),
ip(2), ip(3), ip(4), ip(5))
		
anotherRDD.collect().foreach(println)		
In this case I'm getting the error "java.lang.NumberFormatException: For
input string: ""												



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message