hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Kamath (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-935) Skewed join throws an exception when used with map keys
Date Mon, 31 Aug 2009 22:42:32 GMT

    [ https://issues.apache.org/jira/browse/PIG-935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749664#action_12749664

Pradeep Kamath commented on PIG-935:

Review Comment:
The fix looks good - the only change that would be good to have is in MRCompiler.getSamplingJob()
to explicity check if any field position obtained from the getSortCols() method is -1 and
in those cases throw an "Internal Error setting up sampling job" exception (you may need to
introduce a new error code and follow the error handling guidelines to do this.

> Skewed join throws an exception when used with map keys
> -------------------------------------------------------
>                 Key: PIG-935
>                 URL: https://issues.apache.org/jira/browse/PIG-935
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Sriranjan Manjunath
>         Attachments: skjoinmapbug.patch
> Skewed join throws a runtime exception for the following query:
> A = load 'map.txt' as (e);
> B = load 'map.txt' as (f);
> C = join A by (chararray)e#'a', B by (chararray)f#'a' using "skewed";
> explain C;
> Exception:
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast
cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.PO
> Project
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.getSortCols(MRCompiler.java:1492)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.getSamplingJob(MRCompiler.java:1894)
>         ... 27 more

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message