spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cesar Flores <ces...@gmail.com>
Subject Is coalesce smart while merging partitions?
Date Thu, 08 Oct 2015 02:00:56 GMT
It is my understanding that the default behavior of coalesce function when
the user reduce the number of partitions is to only merge them without
executing shuffle.

My question is: Is this merging smart? For example does spark try to merge
the small partitions first or the election of partitions to merge is random?


Thanks
-- 
Cesar Flores

Mime
View raw message