pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-4670) Embedded Python scripts still parse line by line
Date Fri, 02 Oct 2015 20:12:26 GMT

     [ https://issues.apache.org/jira/browse/PIG-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Rohini Palaniswamy updated PIG-4670:
    Attachment: PIG-4670-2.patch

Actually it is better to have a test case added to verify if it is actually properly working.
Added to the existing testcase which was testing the number of initiations when script was

> Embedded Python scripts still parse line by line
> ------------------------------------------------
>                 Key: PIG-4670
>                 URL: https://issues.apache.org/jira/browse/PIG-4670
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Rohini Palaniswamy
>            Assignee: Rohini Palaniswamy
>             Fix For: 0.16.0
>         Attachments: PIG-4670-1.patch, PIG-4670-2.patch
> PIG-3204 fixed pig script parsing to parse in batches instead of line by line. But the
fix in BoundScript is not right and it is still parsing line by line. That makes parsing take
long time for very large pig scripts using PigStorage when there is no schema file stored
and without -noschema as it tries to find the schema file lots of times.
> It should be grunt.parseStopOnError(false); instead of grunt.parseStopOnError(true);
to make it parse statements in batch.

This message was sent by Atlassian JIRA

View raw message