spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Cheung <>
Subject Re: Do existing R packages work with SparkR data frames
Date Wed, 23 Dec 2015 18:14:10 GMT
SparkR has some support for machine learning algorithm like glm.
For existing R packages, currently you would need to collect to convert into R data.frame
- assuming it fits into the memory of the driver node, though that would be required to work
with R package in any case.

From: Lan <>
Sent: Tuesday, December 22, 2015 4:50 PM
Subject: Do existing R packages work with SparkR data frames
To:  <>

 Is it possible for existing R Machine Learning packages (which work with R   
 data frames) such as bnlearn, to work with SparkR data frames? Or do I need   
 to convert SparkR data frames to R data frames? Is "collect" the function to   
 do the conversion, or how else to do that?   
 Many Thanks,   
 View this message in context:
 Sent from the Apache Spark User List mailing list archive at   
 To unsubscribe, e-mail:   
 For additional commands, e-mail:   

View raw message