spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sachin Singh <sachin.sha...@gmail.com>
Subject Spark SQL implementation error
Date Tue, 30 Dec 2014 09:43:42 GMT
I have a table(csv file) loaded data on that by creating POJO as per table
structure,and created SchemaRDD as under
JavaRDD<Test1> testSchema =
sc.textFile("D:/testTable.csv").map(GetTableData);/* GetTableData will
transform the all table data in testTable object*/
JavaSchemaRDD schemaTest = sqlContext.applySchema(testSchema, Test.class);
		schemaTest.registerTempTable("testTable");

JavaSchemaRDD sqlQuery = sqlContext.sql("SELECT * FROM testTable");
List<String> totDuration = sqlQuery.map(new Function<Row, String>() {
		  public String call(Row row) {
		    return "Field1is : " + row.getInt(0);
		  }
		}).collect();
its working fine
but.........
if I am changing query as(rest code is same)-  JavaSchemaRDD sqlQuery =
sqlContext.sql("SELECT sum(field1) FROM testTable group by field2"); 
error as - Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.spark.rdd.ShuffledRDD.<init>(Lorg/apache/spark/rdd/RDD;Lorg/apache/spark/Partitioner;)V

Please help and Suggest 



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-implementation-error-tp20901.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message