pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koji Noguchi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-4896) Param substitution ignored when redefined
Date Mon, 16 May 2016 21:54:12 GMT

     [ https://issues.apache.org/jira/browse/PIG-4896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Koji Noguchi updated PIG-4896:
------------------------------
    Attachment: pig-4896-v01.patch

Original issue was from PIG-3359 where we added 
{quote}
There's a slight complication with the handling of the warnings in that "val" passed to those
functions in PreprocessorContext is the raw string, which might not be the same as the val
put into param_val. I added a separate hashtable "param_source" to keep track of this original
argument, and if it's seen again to skip it instead of logging the warning.
{quote}

For non-shell {{processOrdLine}}, I've simply took out this "param_source" and compare the
result from "param_value".
For shell {{processShellCmd}}, I've kept the "param_source' but using the value after the
param replacement so that we only use the cache result when command-line matches,

Running test.

> Param substitution ignored when redefined 
> ------------------------------------------
>
>                 Key: PIG-4896
>                 URL: https://issues.apache.org/jira/browse/PIG-4896
>             Project: Pig
>          Issue Type: Bug
>          Components: parser
>            Reporter: Koji Noguchi
>            Priority: Minor
>         Attachments: pig-4896-v01.patch
>
>
> After PIG-3359, the way we cache parameter substitution results changed a bit.
> {code:title=test.pig}
> A = load 'input.txt' ;
> %declare output '/tmp/abc';
> %declare actualoutput '$output.out';
> store A into '$actualoutput';
> %declare output '/tmp/def';
> %declare actualoutput '$output.out';
> store A into '$actualoutput';
> %declare number '1d';
> %declare shellout `bash -c "date -v-$number +'%Y%m%d_%H:%S_%s'; sleep 1" `
> store A into '$shellout';
> %declare shellout `bash -c "date -v-$number +'%Y%m%d_%H:%S_%s'; sleep 1" `
> store A into '$shellout';
> %declare number '2d';
> %declare shellout `bash -c "date -v-$number +'%Y%m%d_%H:%S_%s'; sleep 1" `
> store A into '$shellout';
> {code}
> Result from pig 0.11 (almost correct)
> {panel}
> A = load 'input.txt' ;
> store A into '/tmp/abc.out';
> store A into '/tmp/def.out';
> store A into '20160515_17:32_1463346752';
> store A into '20160515_17:33_1463346753'; {color:red}NO{color}
> store A into '20160514_17:34_1463260354';
> {panel}
> Result from trunk 
> {panel}
> A = load 'input.txt' ;
> store A into '/tmp/abc.out';
> store A into '/tmp/abc.out';  {color:red} NO{color}
> store A into '20160515_15:10_1463338810';
> store A into '20160515_15:10_1463338810';
> store A into '20160515_15:10_1463338810'; {color:red}NO{color}
> {panel}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message