spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From icexelloss <>
Subject [GitHub] spark issue #19872: WIP: [SPARK-22274][PySpark] User-defined aggregation fun...
Date Fri, 08 Dec 2017 00:47:52 GMT
Github user icexelloss commented on the issue:
    And to @holdenk 's question. Pandas group_agg udf fundamentally uses different physical
plan than the existing java/scala udf and therefore it's hard to combine them together. I
don't know a good way to do this, the closest is maybe to compute java/scala and python aggregation
separately and join them together.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message