spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tal Grynbaum <tal.grynb...@gmail.com>
Subject parsing embedded json in spark
Date Thu, 22 Dec 2016 07:42:20 GMT
Hi,

I have a dataframe that contain an embedded json string in one of the fields
I'd tried to write a UDF function that will parse it using lift-json, but
it seems to take a very long time to process, and it seems that only the
master node is working.

Has anyone dealt with such a scenario before and can give me some hints?

Thanks
Tal

Mime
View raw message