spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nayan sharma <nayansharm...@gmail.com>
Subject splitting columns into new columns
Date Sun, 16 Jul 2017 18:25:17 GMT
I’ve a Dataframe where in some columns there are multiple values, always separated by ^

phone|contact|
ERN~58XXXXXX7~^EPN~5XXXXX551~|C~MXXX~MSO~^CAxxE~~~~~~3XXX5|

phone1|phone2|contact1|contact2| 
ERN~5XXXXXXX7|EPN~58XXXX91551~|C~MXXXH~MSO~|CAxxE~~~~~~3XXX5|
How can this be achieved using loop as the separator between column values
are not constant.

data.withColumn("phone",split($"phone","\\^")).select($"phon‌​e".getItem(0).as("ph‌​one1"),$"phone".getI‌​tem(1).as("phone2”))
 I though of doing this way but the problem is  column are having 100+ separator between the
column values



Thank you,
Nayan
Mime
View raw message