pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koji Noguchi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3266) Pig takes forever to parse scripts with foreach + multi level binconds
Date Tue, 02 Apr 2013 18:17:16 GMT

    [ https://issues.apache.org/jira/browse/PIG-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620098#comment-13620098
] 

Koji Noguchi commented on PIG-3266:
-----------------------------------

If I revert the change from PIG:1387, parsing speed comes back to 90 seconds (pig-0.9 level)

{noformat}
src/org/apache/pig/parser/QueryParser.g
-projectable_expr: func_eval | col_ref | bin_expr | type_conversion
+projectable_expr: func_eval | col_ref | bin_expr
{noformat}

I don't know anything about antlr, but I guess it cannot tell whether the given tokens are
bin_expr or type_conversion when starting with '(' so spending extra cycles to check both.
                
> Pig takes forever to parse scripts with foreach + multi level binconds 
> -----------------------------------------------------------------------
>
>                 Key: PIG-3266
>                 URL: https://issues.apache.org/jira/browse/PIG-3266
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.10.0, 0.11
>            Reporter: Koji Noguchi
>
> Following pig script parsing takes 
> *  1 second  in pig-0.8
> * 90 seconds in pig-0.9
> * forever in pig-0.10 (it's taking literally hours)
> {noformat}
> A = load 'input.txt' as (mynum:float, mychar:chararray);
> B = foreach A generate mychar,
> (mynum < 0 ? 0 :
> (mynum < 1 ? 1 :
> (mynum < 2 ? 2 :
> (mynum < 3 ? 3 :
> (mynum < 4 ? 4 :
> (mynum < 5 ? 5 :
> (mynum < 6 ? 6 :
> (mynum < 7 ? 7 :
> (mynum < 8 ? 8 :
> (mynum < 9 ? 9 :
> (mynum < 10 ? 10 :
> (mynum < 11 ? 11 :
> (mynum < 12 ? 12 :
> (mynum < 13 ? 13 :
> (mynum < 14 ? 14 :
> (mynum < 15 ? 15 :
> (mynum < 16 ? 16 :
> (mynum < 17 ? 17 :
> (mynum < 18 ? 18 :
> (mynum < 19 ? 19 :
> (mynum < 20 ? 20 : 21)))))))))))))))))))));
> dump A;
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message