It is my understanding that the default behavior of coalesce function when the user reduce the number of partitions is to only merge them without executing shuffle.

My question is: Is this merging smart? For example does spark try to merge the small partitions first or the election of partitions to merge is random?

Cesar Flores