spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koert Kuipers <ko...@tresata.com>
Subject in joins, does one side stream?
Date Thu, 17 Sep 2015 18:21:38 GMT
in scalding we join with the smaller side on the left, since the smaller
side will get buffered while the bigger side streams through the join.

looking at CoGroupedRDD i do not get the impression such a distiction is
made. it seems both sided are put into a map that can spill to disk. is
this correct?

thanks

Mime
View raw message