Return-Path: X-Original-To: apmail-pig-dev-archive@www.apache.org Delivered-To: apmail-pig-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4F87C10175 for ; Fri, 4 Oct 2013 00:29:44 +0000 (UTC) Received: (qmail 19425 invoked by uid 500); 4 Oct 2013 00:29:43 -0000 Delivered-To: apmail-pig-dev-archive@pig.apache.org Received: (qmail 19389 invoked by uid 500); 4 Oct 2013 00:29:43 -0000 Mailing-List: contact dev-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list dev@pig.apache.org Received: (qmail 19380 invoked by uid 500); 4 Oct 2013 00:29:43 -0000 Delivered-To: apmail-hadoop-pig-dev@hadoop.apache.org Received: (qmail 19377 invoked by uid 99); 4 Oct 2013 00:29:43 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Oct 2013 00:29:43 +0000 Date: Fri, 4 Oct 2013 00:29:43 +0000 (UTC) From: "Cheolsoo Park (JIRA)" To: pig-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (PIG-3500) Initial implementation of TezCompiler MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PIG-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3500: ------------------------------- Attachment: PIG-3500-1.patch Attached includes an initial version of TezCompiler with unit tests. Note that it query #3 is compiled into 3 Tez vertices (two input vertices and one join vertex) unlike MR plan. The unit test can run with ant test clean -Dtestcase=TestTezCompiler. > Initial implementation of TezCompiler > ------------------------------------- > > Key: PIG-3500 > URL: https://issues.apache.org/jira/browse/PIG-3500 > Project: Pig > Issue Type: Sub-task > Components: tez > Affects Versions: tez-branch > Reporter: Cheolsoo Park > Assignee: Cheolsoo Park > Fix For: tez-branch > > Attachments: PIG-3500-1.patch > > > Implement TezCompiler that compiles physical plan into tez plan. To begin with, we can implement the initial version that works for basic queries as follows: > # Load-Filter-Store > {code} > a = load 'file:///tmp/input' as (x:int, y:int); > b = filter a by x > 0; > c = foreach b generate y; > store c into 'file:///tmp/output'; > {code} > # Load-Filter-GroupBy-Store > {code} > a = load 'file:///tmp/input' as (x:int, y:int); > b = group a by x; > c = foreach b generate group, a; > store c into 'file:///tmp/output'; > {code} > # Load1-Load2-Join-Store > {code} > a = load 'file:///tmp/input1' as (x:int, y:int); > b = load 'file:///tmp/input2' as (x:int, z:int); > c = join a by x, b by x; > d = foreach c generate a::x as x, y, z; > store d into 'file:///tmp/output'; > {code} -- This message was sent by Atlassian JIRA (v6.1#6144)