spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhisheks <>
Subject SQLcontext changing String field to Long
Date Fri, 09 Oct 2015 23:55:11 GMT
Hi there,

I have saved my records in to parquet format and am using Spark1.5. But when
I try to fetch the columns it throws exception*
java.lang.ClassCastException: java.lang.Long cannot be cast to

This filed is saved as String while writing parquet. so here is the sample
code and output for the same.."troubling thing is ::" +
DataFrame df= sqlContext.sql(fileSelectQuery);
JavaRDD<Row> rdd2 = df.toJavaRDD();

First Line in the code (Logger) prints this: 
troubling thing is ::StructType(StructField(batch_id,StringType,true))

But the moment after it the execption comes up. 

Any idea why it is treating the filed as Long? (yeah one unique thing about
column is it is always a number e.g. Time-stamp).

Any help is appreciated.

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message