hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-1336) Optimize POStore serialized into JobConf
Date Wed, 31 Mar 2010 00:46:27 GMT

     [ https://issues.apache.org/jira/browse/PIG-1336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Daniel Dai updated PIG-1336:

    Status: Patch Available  (was: Open)

Good catch. Thanks Richard. Modify the patch to address that.

> Optimize POStore serialized into JobConf
> ----------------------------------------
>                 Key: PIG-1336
>                 URL: https://issues.apache.org/jira/browse/PIG-1336
>             Project: Pig
>          Issue Type: Improvement
>          Components: impl
>    Affects Versions: 0.7.0
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>         Attachments: PIG-1336-1.patch, PIG-1336-2.patch, PIG-1336-3.patch, PIG-1336-4.patch
> We serialize POStore too early in the JobControlCompiler. At that time, storeFunc have
unconstraint link to other operator; in the worst case, it will chain the whole physical plan.
Also, in multi-store case, POStore has link to its data source, which is not needed and will
increase the footprint of serialized POStore. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message