spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koert Kuipers <ko...@tresata.com>
Subject Re: Dataset Question: No Encoder found for Set[(scala.Long, scala.Long)]
Date Wed, 01 Feb 2017 01:04:35 GMT
set is currently not supported. you can use kryo encoder. there is no other
work around that i know of.

import org.apache.spark.sql.{ Encoder, Encoders }
implicit def setEncoder[X]: Encoder[Set[X]] = Encoders.kryo[Set[X]]

On Tue, Jan 31, 2017 at 7:33 PM, Jerry Lam <chilinglam@gmail.com> wrote:

> Hi guys,
>
> I got an exception like the following, when I tried to implement a user
> defined aggregation function.
>
>  Exception in thread "main" java.lang.UnsupportedOperationException: No
> Encoder found for Set[(scala.Long, scala.Long)]
>
> The Set[(Long, Long)] is a field in the case class which is the output
> type for the aggregation.
>
> Is there a workaround for this?
>
> Best Regards,
>
> Jerry
>

Mime
View raw message