hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-336) NULL checks are not in place in the types branch
Date Wed, 06 Aug 2008 01:20:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620110#action_12620110

Alan Gates commented on PIG-336:

This patch looks great.  What a pile of work!

I have one comment.  The KeyTypeDiscoverVisitor is used in the map and combine sections PigMapReduce
and PigCombiner.  This should be done sooner.  Those classes are focussed on actual execution,
not execution planning.  I think you're waiting until then because you only want to visit
specific pieces of the physical plan in each map, combine, or reduce phase.  If that's the
case, you should instead run this visitor in MapReduceLauncher and LocalLauncher, where the
physical plan is compiled into a map reduce plan.  (Look for CombinerOptimizer, you should
run the KeyTypeDiscoveryVisitor right after that.)  You can then record that information in
the MapReduceOper itself, and read that in PigMapReduce and PigCombiner.

> NULL checks are not in place in the types branch
> ------------------------------------------------
>                 Key: PIG-336
>                 URL: https://issues.apache.org/jira/browse/PIG-336
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>            Assignee: Pradeep Kamath
>            Priority: Critical
>             Fix For: types_branch
>         Attachments: PIG-336-part1.patch, PIG-336-part1_v2.patch, PIG-336-part1_v3.patch,
> The following code currently does not work
> {code}
> B = filter A by $0 is null and $1 is null;
> {code}
> Some other things which don't work with nulls include POAND, POOR etc

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message