spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <>
Subject RE: How to calculate percentile of a column of DataFrame?
Date Fri, 09 Oct 2015 19:09:09 GMT
Where can we find other available functions such as lit() ? I can’t find lit in the api.


From: Michael Armbrust []
Sent: Friday, October 09, 2015 4:04 PM
To: unk1102
Cc: user
Subject: Re: How to calculate percentile of a column of DataFrame?

You can use callUDF(col("mycol"), lit(0.25)) to call hive UDFs from dataframes.

On Fri, Oct 9, 2015 at 12:01 PM, unk1102 <<>>
Hi how to calculate percentile of a column in a DataFrame? I cant find any
percentile_approx function in Spark aggregation functions. For e.g. in Hive
we have percentile_approx and we can use it in the following way

hiveContext.sql("select percentile_approx("mycol",0.25) from myTable);

I can see ntile function but not sure how it is gonna give results same as
above query please guide.

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:<>
For additional commands, e-mail:<>

View raw message