See the documentation for the options given to the csv function:
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameReader@csv(paths:String*):org.apache.spark.sql.DataFrame
The options can be passed with the option/options functions to the
DataFrameReader class (a similar syntax is also available in pySpark).
-Nick
Nicholas Szandor Hakobian, Ph.D.
Senior Data Scientist
Rally Health
nicholas.hakobian@rallyhealth.com
On Sat, Dec 31, 2016 at 9:58 AM, A Shaikh <shaikh.afzal@gmail.com> wrote:
> In Pyspark 2 loading file wtih any delimiter into Dataframe is pretty
> straightforward
> spark.read.csv(file, schema=, sep='|')
>
> Is there something similar in Spark 2 in Scala! spark.read.csv(path,
> sep='|')?
>
>
|