spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Meethu Mathew <meethu.mat...@flytxt.com>
Subject Use of MapConverter, ListConverter in python to java object conversion
Date Tue, 13 Jan 2015 12:53:34 GMT
Hi all,

In the python object to java conversion done in the method _py2java in 
spark/python/pyspark/mllib/common.py, why  we are doing individual 
conversion  using MpaConverter,ListConverter? The same can be acheived 
using

bytearray(PickleSerializer().dumps(obj))
obj = sc._jvm.SerDe.loads(bytes)

Is there any performance gain or something in using individual 
converters rather than PickleSerializer?

-- 

Regards,

*Meethu*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message