hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-1277) Pig should give error message when cogroup on tuple keys of different inner type
Date Mon, 12 Jul 2010 20:47:54 GMT

     [ https://issues.apache.org/jira/browse/PIG-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Olga Natkovich updated PIG-1277:
--------------------------------

    Fix Version/s: 0.9.0

> Pig should give error message when cogroup on tuple keys of different inner type
> --------------------------------------------------------------------------------
>
>                 Key: PIG-1277
>                 URL: https://issues.apache.org/jira/browse/PIG-1277
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.6.0
>            Reporter: Daniel Dai
>             Fix For: 0.9.0
>
>
> When we cogroup on a tuple, if the inner type of tuple does not match, we treat them
as different keys. This is confusing. It is desirable to give error/warnings when it happens.
> Here is one example:
> UDF:
> {code}
> public class MapGenerate extends EvalFunc<Map> {
>     @Override
>     public Map exec(Tuple input) throws IOException {
>         // TODO Auto-generated method stub
>         Map m = new HashMap();
>         m.put("key", new Integer(input.size()));
>         return m;
>     }
>     
>     @Override
>     public Schema outputSchema(Schema input) {
>         return new Schema(new Schema.FieldSchema(null, DataType.MAP));
>     }
> }
> {code}
> Pig script: 
> {code}
> a = load '1.txt' as (a0);
> b = foreach a generate a0, MapGenerate(*) as m:map[];
> c = foreach b generate a0, m#'key' as key;
> d = load '2.txt' as (c0, c1);
> e = cogroup c by (a0, key), d by (c0, c1);
> dump e;
> {code}
> 1.txt
> {code}
> 1
> {code}
> 2.txt
> {code}
> 1 1
> {code}
> User expected result (which is not right):
> {code}
> ((1,1),{(1,1)},{(1,1)})
> {code}
> Real result:
> {code}
> ((1,1),{(1,1)},{})
> ((1,1),{},{(1,1)})
> {code}
> We shall give user the message that we can not merge the key due to the type mismatch.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message