Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 4FF8B200B4B for ; Thu, 7 Jul 2016 07:22:13 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 4E75C160A64; Thu, 7 Jul 2016 05:22:13 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 94CC4160A74 for ; Thu, 7 Jul 2016 07:22:12 +0200 (CEST) Received: (qmail 38717 invoked by uid 500); 7 Jul 2016 05:22:11 -0000 Mailing-List: contact dev-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list dev@pig.apache.org Received: (qmail 38661 invoked by uid 500); 7 Jul 2016 05:22:11 -0000 Delivered-To: apmail-hadoop-pig-dev@hadoop.apache.org Received: (qmail 38658 invoked by uid 99); 7 Jul 2016 05:22:11 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Jul 2016 05:22:11 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 1412D2C02AD for ; Thu, 7 Jul 2016 05:22:11 +0000 (UTC) Date: Thu, 7 Jul 2016 05:22:11 +0000 (UTC) From: "Julien Le Dem (JIRA)" To: pig-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (PIG-3764) Compile physical operators to bytecode MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 07 Jul 2016 05:22:13 -0000 [ https://issues.apache.org/jira/browse/PIG-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365633#comment-15365633 ] Julien Le Dem commented on PIG-3764: ------------------------------------ [~rohini] This sounds good. It does not have to be totally inlined since the JIT will inline method calls, you want to avoid virtual calls though. My prototype is still out there [1]. One thing it did not take into account is nulls. But I think this can be branch out separately (evaluate ignoring the nulls and then evaluate the is null) Generating asm directly can be unwieldy. That's why I had made Brennus [2] to factor out a lot of the logic (different operations per type, different stack frame size per type, all sorts of special cases) see proto. [1] 1: https://github.com/julienledem/pig/compare/trunk...compile_physical_plan 2: https://github.com/julienledem/brennus > Compile physical operators to bytecode > -------------------------------------- > > Key: PIG-3764 > URL: https://issues.apache.org/jira/browse/PIG-3764 > Project: Pig > Issue Type: Improvement > Components: impl > Reporter: Julien Le Dem > Labels: GSOC2014 > > I started a prototype here: > https://github.com/julienledem/pig/compare/trunk...compile_physical_plan > The current physical plan is relatively inefficient at evaluating expressions. > In the context of a better execution engine (Tez, Spark, ...), compiling expressions to bytecode would be a significant speedup. > This is a candidate project for Google summer of code 2014. More information about the program can be found at https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v6.3.4#6332)