pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Bain (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3536) Make distinct work
Date Tue, 05 Nov 2013 01:02:17 GMT

    [ https://issues.apache.org/jira/browse/PIG-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13813499#comment-13813499
] 

Alex Bain commented on PIG-3536:
--------------------------------

Notes from my ReviewBoard post:
===========================
Implement DISTINCT for Pig-on-Tez by providing a (very straightforward) implementation in
TezCompiler.java.

For the moment, this does NOT use two optimizations done in the MRCompiler. We will create
a separate JIRA for these optimizations:
1. A distinct combiner
2. A combiner optimizer that replaces certain uses of DISTINCT with an algebraic udf

This patch includes:
-A unit test in TestTezCompiler.java
-An e2e test

> Make distinct work
> ------------------
>
>                 Key: PIG-3536
>                 URL: https://issues.apache.org/jira/browse/PIG-3536
>             Project: Pig
>          Issue Type: Sub-task
>          Components: tez
>    Affects Versions: tez-branch
>            Reporter: Cheolsoo Park
>            Assignee: Alex Bain
>             Fix For: tez-branch
>
>         Attachments: PIG-3536.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message