pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1787) Error in logical plan generated
Date Mon, 10 Jan 2011 23:43:47 GMT

    [ https://issues.apache.org/jira/browse/PIG-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12979854#action_12979854
] 

Daniel Dai commented on PIG-1787:
---------------------------------

Note the test case only works in mapreduce mode. In local mode, parallel 2 is not grantted.


> Error in logical plan generated
> -------------------------------
>
>                 Key: PIG-1787
>                 URL: https://issues.apache.org/jira/browse/PIG-1787
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.0
>            Reporter: Anitha Raju
>            Assignee: Daniel Dai
>         Attachments: PIG-1787-1.patch, PIG-1787-2.patch
>
>
> Here is a sample pig script:
> set default_parallel 2
> ALLDATA = load 'sample.txt' using PigStorage() as (id, spaceid, type, pcid);
> C1 = filter ALLDATA by (type == 'p' and
>                    (spaceid == '1196250013'
>                     or spaceid == '1196250024'
>                     or spaceid == '1196250011'));
> C2 = group C1 by pcid;
> C3 = foreach C2 generate flatten(group) as (pc_id), COUNT(C1) as tot;
> C4 = order C3 by tot desc;
> C5 = limit C4 3;
> C6 = join C5 by pc_id, C1 by pcid;
> dump C6;
> sample.txt:
> 1       1196250013      p       1234
> 2       1196250024      p       2314
> 3       1196250011      t       1111
> 4       1111111111      p       1231
> 5       1196250013      p       1254
> 6       1196250024      p       9007
> This fails with the error 
> java.io.IOException: Type mismatch in key from map: expected org.apache.pig.impl.io.NullableLongWritable,
recieved
> org.apache.pig.impl.io.NullableBytesWritable
> when both pc_id and pcid are of type bytearray.
> The script seems to work when 
> 	a) replicated join is substituted in the place of the regular join 
> 	b) pcid is cast to long in the loader 
> 	c) doing a dump of any statement before C6
> 	d) setting default_parallel to 1 or removing it.
> 	
> One possible cause seems to be with the logical plan generation during the projection
operation in C4 as can be observed from the describe statement. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message