pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3347) Store invocation brings side effect
Date Tue, 04 Feb 2014 20:44:09 GMT

    [ https://issues.apache.org/jira/browse/PIG-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13891178#comment-13891178
] 

Daniel Dai commented on PIG-3347:
---------------------------------

It could but introduce a lot of complications. Currently only LOForEach/LOSplitOutput is dealing
with dup-uid, otherwise it will sprawl to all operators and all optimizer rules.

> Store invocation brings side effect
> -----------------------------------
>
>                 Key: PIG-3347
>                 URL: https://issues.apache.org/jira/browse/PIG-3347
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt
>    Affects Versions: 0.11
>         Environment: local mode
>            Reporter: Sergey
>            Assignee: Daniel Dai
>            Priority: Critical
>             Fix For: 0.12.1
>
>         Attachments: PIG-3347-1.patch, PIG-3347-2-testonly.patch, PIG-3347-3.patch, PIG-3347-4-testonly.patch,
PIG-3347-5.patch
>
>
> The problem is that intermediate 'store' invocation "changes" the final store output.
Looks like it brings some kind of side effect. We did use 'local' mode to run script
> here is the input data:
> 1
> 1
> Here is the script:
> {code}
> a = load 'test';
> a_group = group a by $0;
> b = foreach a_group {
>   a_distinct = distinct a.$0;
>   generate group, a_distinct;
> }
> --store b into 'b';
> c = filter b by SIZE(a_distinct) == 1;
> store c into 'out';
> {code}
> We expect output to be:
> 1 1
> The output is empty file.
> Uncomment {code}--store b into 'b';{code} line and see the diffrence.
> Yuo would get expected output.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message