pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2442) Multiple Stores in pig streaming causes infinite waiting
Date Tue, 20 Mar 2012 23:50:39 GMT

    [ https://issues.apache.org/jira/browse/PIG-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233957#comment-13233957
] 

Thejas M Nair commented on PIG-2442:
------------------------------------

+1 . Can you make this change before commit ? - 
In POSplit.getStreamCloseResult
{code}
if (inpEOP && parentPlan.endOfAllInput) {
{code}

can be changed to 
{code}
if (inpEOP ) {
{code}
as the function is called only if parentPlan.endOfAllInput is true. 

                
> Multiple Stores in pig streaming causes infinite waiting
> --------------------------------------------------------
>
>                 Key: PIG-2442
>                 URL: https://issues.apache.org/jira/browse/PIG-2442
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.1, 0.9.0
>            Reporter: Anitha Raju
>            Assignee: xuting zhao
>             Fix For: 0.10, 0.11
>
>         Attachments: PIG-2442-1.patch
>
>
> Hi,
> If there are multiple store in a pig streaming script, it goes into infinite waiting.

> Script
> {code}
> DEFINE SCRIPT `./a.pl` SHIP ('/homes/anithar/a.pl');;
> DEFINE SCRIPT1 `./b.pl` SHIP ('/homes/anithar/b.pl');;
> A = LOAD 'test.txt' USING PigStorage() ;
> B1 = STREAM A THROUGH SCRIPT ;
> B1 = foreach B1 generate $0;
> STORE B1 INTO 'B1' USING PigStorage();
> B2 =  STREAM B1 THROUGH SCRIPT1;
> STORE B2 INTO 'B2' USING PigStorage();
> {code}
> a.pl
> --------
> #! /usr/bin/perl -w
> while (my $line = <STDIN>) {
>         print uc($line);
> }
> --------
> b.pl
> ---------
> #! /usr/bin/perl -w
> while (my $line = <STDIN>) {
>         print $line;
> }
> ---------
> Input (test.txt)
> {code}
> test
> hi
> hello
> {code}
> This infinite waiting happens randomly causing the job to fail with "Task attempt failed
to report
> status for 605 seconds. Killing!". 
> Same happens with 0.8 version too.
> Regards,
> Anitha

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message