spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From KhajaAsmath Mohammed <>
Subject Write only one output file in Spark SQL
Date Fri, 11 Aug 2017 15:52:58 GMT

I am using spark sql to write data back to hdfs and it is resulting in
multiple output files.

I tried changing number spark.sql.shuffle.partitions=1 but it resulted in
very slow performance.

Also tried coalesce and repartition still the same issue. any suggestions?



View raw message