pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-1336) Optimize POStore serialized into JobConf
Date Thu, 01 Apr 2010 22:52:27 GMT

     [ https://issues.apache.org/jira/browse/PIG-1336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Daniel Dai updated PIG-1336:

       Resolution: Fixed
    Fix Version/s: 0.7.0
     Hadoop Flags: [Reviewed]
           Status: Resolved  (was: Patch Available)

This issue is an optimization and is hard to write a unit test. Test it manually and it works.
Patch committed to both trunk and 0.7 branch.

> Optimize POStore serialized into JobConf
> ----------------------------------------
>                 Key: PIG-1336
>                 URL: https://issues.apache.org/jira/browse/PIG-1336
>             Project: Pig
>          Issue Type: Improvement
>          Components: impl
>    Affects Versions: 0.7.0
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.7.0
>         Attachments: PIG-1336-1.patch, PIG-1336-2.patch, PIG-1336-3.patch, PIG-1336-4.patch
> We serialize POStore too early in the JobControlCompiler. At that time, storeFunc have
unconstraint link to other operator; in the worst case, it will chain the whole physical plan.
Also, in multi-store case, POStore has link to its data source, which is not needed and will
increase the footprint of serialized POStore. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message