spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From KhajaAsmath Mohammed <mdkhajaasm...@gmail.com>
Subject Write only one output file in Spark SQL
Date Fri, 11 Aug 2017 15:52:58 GMT
Hi,



I am using spark sql to write data back to hdfs and it is resulting in
multiple output files.



I tried changing number spark.sql.shuffle.partitions=1 but it resulted in
very slow performance.



Also tried coalesce and repartition still the same issue. any suggestions?



Thanks,

Asmath

Mime
View raw message