ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Taras Ledkov <tled...@gridgain.com>
Subject Re: IgniteRDD type mapping
Date Tue, 14 Jun 2016 08:32:21 GMT
Alexey,

According to the Spark type mapping StructType is mapped to the
org.apache.spark.sql.Row class. Please take a look at the Spark docs
(https://spark.apache.org/docs/1.3.0/sql-programming-guide.html#data-types)

or at the source code:
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala#L251

You filed the issue ignite-3077 against this.

I mistook  about the root cause. The information about the class of the 
object is helpless.

Spark supports only primitive types and some popular classes to 
represent decimal & dates, Map, List and the its own class the Row to 
represent complex objects.

09.06.2016 3:43, Alexey Goncharuk пишет:
> Taras,
>
> According to IgniteRDD non-predefined types (including Object) are mapped
> to StructType with no fields, which should be recognizable by Spark. Can
> you explain why MatchError happens?
> ​
>

-- 
Taras Ledkov
Mail-To: tledkov@gridgain.com


Mime
View raw message