spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: Returning numpy types from udfs
Date Sat, 05 Dec 2015 15:08:25 GMT
Not aware of any jira ticket, but it does sound like a great idea.


On Sat, Dec 5, 2015 at 11:03 PM, Justin Uang <justin.uang@gmail.com> wrote:

> Hi,
>
> I have fallen into the trap of returning numpy types from udfs, such as
> np.float64 and np.int. It's hard to find the issue because they behave
> pretty much as regular pure Python floats and doubles, so can we make
> PYSPARK automatically translate them?
>
> If so, I'll create a Jira ticket.
>
> Justin
>

Mime
View raw message