Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 98423200D20 for ; Tue, 17 Oct 2017 16:29:16 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 94AA81609DE; Tue, 17 Oct 2017 14:29:16 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id DB9651609D7 for ; Tue, 17 Oct 2017 16:29:15 +0200 (CEST) Received: (qmail 70760 invoked by uid 500); 17 Oct 2017 14:29:15 -0000 Mailing-List: contact reviews-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list reviews@spark.apache.org Received: (qmail 70749 invoked by uid 99); 17 Oct 2017 14:29:14 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Oct 2017 14:29:14 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id BD123DFBC7; Tue, 17 Oct 2017 14:29:14 +0000 (UTC) From: ueshin To: reviews@spark.apache.org Reply-To: reviews@spark.apache.org Message-ID: Subject: [GitHub] spark pull request #19517: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().... Content-Type: text/plain Date: Tue, 17 Oct 2017 14:29:14 +0000 (UTC) archived-at: Tue, 17 Oct 2017 14:29:16 -0000 GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/19517 [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply() with pandas udf ## What changes were proposed in this pull request? This is a follow-up of #18732. This pr modifies `GroupedData.apply()` method to convert pandas udf to grouped udf implicitly. ## How was this patch tested? Exisiting tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ueshin/apache-spark issues/SPARK-20396/fup2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/19517.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #19517 ---- commit 4d2bd959e1eeabb4f72cfbb52a374ce721030507 Author: Takuya UESHIN Date: 2017-10-16T06:45:55Z Introduce `@pandas_grouped_udf` decorator for grouped vectorized UDF. commit f0968702038e11c9c9a8f305c61f72d3f9e00f9a Author: Takuya UESHIN Date: 2017-10-16T08:03:30Z Use PythonUdfType instead of vectorized and grouped. commit 639af2cee77456271d5f2f536d4712ab8e01a89d Author: Takuya UESHIN Date: 2017-10-16T13:42:58Z Update an error message. commit 10512a64a9560eee6d3f65802abd042dedf0cafb Author: Takuya UESHIN Date: 2017-10-16T13:43:51Z Add a test to use data type string. commit 789e642763ab4f59e14137fcc75b514223bc7aae Author: Takuya UESHIN Date: 2017-10-16T14:13:43Z Restrict the number of arguments for grouped udf to only 1. commit 122a7bccaff11def2c12cfccdd00244394ed3478 Author: Takuya UESHIN Date: 2017-10-16T16:24:03Z Restrict checking the number of arguments. commit fdafb3561d44ca2583380b7aeaf7843ce5285b1e Author: Takuya UESHIN Date: 2017-10-16T16:54:23Z Revert "Restrict checking the number of arguments." This reverts commit 122a7bccaff11def2c12cfccdd00244394ed3478. commit 94d05f4f8d5c663319ec12668dbd1206ffa2e83a Author: Takuya UESHIN Date: 2017-10-16T18:10:50Z Address comments. commit 733296951b45d760aa0a8465eb0189077ea67372 Author: Takuya UESHIN Date: 2017-10-16T18:33:08Z Add tests for unsupported type. commit 85f250d0eda56606a599c5fb15046ef0fd63a3c4 Author: Takuya UESHIN Date: 2017-10-17T04:59:34Z Address a comment. commit 7b386c4be48c0a2e8de6f04cf341de13e8e98444 Author: Takuya UESHIN Date: 2017-10-17T14:12:37Z Remove `@pandas_grouped_udf` and convert implicitly. ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org For additional commands, e-mail: reviews-help@spark.apache.org