spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From icexelloss <...@git.apache.org>
Subject [GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...
Date Tue, 10 Oct 2017 18:15:48 GMT
Github user icexelloss commented on a diff in the pull request:

    https://github.com/apache/spark/pull/18732#discussion_r143810539
  
    --- Diff: python/pyspark/sql/group.py ---
    @@ -192,7 +193,84 @@ def pivot(self, pivot_col, values=None):
                 jgd = self._jgd.pivot(pivot_col)
             else:
                 jgd = self._jgd.pivot(pivot_col, values)
    -        return GroupedData(jgd, self.sql_ctx)
    +        return GroupedData(jgd, self._df)
    +
    +    @since(2.3)
    +    def apply(self, udf):
    +        """
    +        Maps each group of the current :class:`DataFrame` using a pandas udf and returns
the result
    +        as a `DataFrame`.
    +
    +        The user-defined function should take a `pandas.DataFrame` and return another
    +        `pandas.DataFrame`. For each group, all columns are passed together as a `pandas.DataFrame`
    +        to the user-function and the returned `pandas.DataFrame` are combined as a `DataFrame`.
    --- End diff --
    
    Fixed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message