spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olivier Girardot <o.girar...@lateral-thoughts.com>
Subject Re: Spark SQL and Kryo registration
Date Thu, 18 Aug 2016 06:16:59 GMT
Hi everyone, it seems that it works now out of the box. So nevermind, registration is compatible
with spark 2.0 when using dataframes.
Regards,
Olivier.





On Fri, Aug 5, 2016 10:07 AM, Maciej Bryński maciek@brynski.pl wrote:
Hi Olivier, Did you check performance of Kryo ? I have observations that Kryo is slightly
slower than Java Serializer.
Regards, Maciek
2016-08-04 17:41 GMT+02:00 Amit Sela < amitsela33@gmail.com > :
It should. Codegen uses the SparkConf in SparkEnv when instantiating a new
Serializer.
On Thu, Aug 4, 2016 at 6:14 PM Jacek Laskowski < jacek@japila.pl > wrote:
Hi Olivier,

I don't know either, but am curious what you've tried already.

Jacek


On 3 Aug 2016 10:50 a.m., "Olivier Girardot" < o.girardot@lateral-thoughts. com > wrote:
Hi everyone, I'm currently to use Spark 2.0.0 and making Dataframes work with kryo. registrationRequired=true
Is it even possible at all considering the codegen ?
Regards,
Olivier Girardot | Associé
o.girardot@lateral-thoughts. com
+33 6 24 09 17 94


--
Maciek Bryński

Olivier Girardot | Associé
o.girardot@lateral-thoughts.com
+33 6 24 09 17 94
Mime
View raw message