Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 3CB8A200B42 for ; Sun, 26 Jun 2016 06:00:43 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 3B421160A67; Sun, 26 Jun 2016 04:00:43 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id A9207160A66 for ; Sun, 26 Jun 2016 06:00:42 +0200 (CEST) Received: (qmail 36293 invoked by uid 500); 26 Jun 2016 04:00:40 -0000 Mailing-List: contact issues-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@spark.apache.org Received: (qmail 36265 invoked by uid 99); 26 Jun 2016 04:00:40 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 26 Jun 2016 04:00:40 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 6D13A2C033A for ; Sun, 26 Jun 2016 04:00:40 +0000 (UTC) Date: Sun, 26 Jun 2016 04:00:40 +0000 (UTC) From: "Kazuaki Ishizaki (JIRA)" To: issues@spark.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (SPARK-16213) Reduce runtime overhead of a program that creates an primitive array in DataFrame MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Sun, 26 Jun 2016 04:00:43 -0000 Kazuaki Ishizaki created SPARK-16213: ---------------------------------------- Summary: Reduce runtime overhead of a program that creates an primitive array in DataFrame Key: SPARK-16213 URL: https://issues.apache.org/jira/browse/SPARK-16213 Project: Spark Issue Type: Improvement Components: SQL Reporter: Kazuaki Ishizaki Reduce runtime overhead of a program that creates an primitive array in DataFrame When a program creates an array in DataFrame, the code generator creates boxing operations. If an array is for primitive type, there are some opportunities for optimizations in generated code to reduce runtime overhead. Here is a simple example that has generated code with boxing operation {code} val df = sparkContext.parallelize(Seq(0.0d, 1.0d), 1).toDF df.selectExpr("Array(value + 1.1d, value + 2.2d)").show {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org For additional commands, e-mail: issues-help@spark.apache.org