spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Meethu Mathew <>
Subject Use of MapConverter, ListConverter in python to java object conversion
Date Tue, 13 Jan 2015 12:53:34 GMT
Hi all,

In the python object to java conversion done in the method _py2java in 
spark/python/pyspark/mllib/, why  we are doing individual 
conversion  using MpaConverter,ListConverter? The same can be acheived 

obj = sc._jvm.SerDe.loads(bytes)

Is there any performance gain or something in using individual 
converters rather than PickleSerializer?




  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message