pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prashant Kommireddi (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-798) Schema errors when using PigStorage and none when using BinStorage in FOREACH??
Date Tue, 29 Nov 2011 10:43:40 GMT

    [ https://issues.apache.org/jira/browse/PIG-798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159188#comment-13159188
] 

Prashant Kommireddi commented on PIG-798:
-----------------------------------------

Thanks Daniel. Unfortunately my UDF cannot return a numeric type as the return value could
non-numeric as well. The value depends on the arguments I pass to the function.

For Error 2: how is it that if my UDF just returned a single chararray instead of a tuple
of chararrays the SUM function would work? For eg, 
{code}
A = LOAD 'input.dat'; 
--assuming MYUDF(*,'id') always returns numeric value
B = FOREACH A GENERATE (int)MYUDF(*, 'id') as id;
C = GROUP B BY $0;
D = FOREACH C GENERATE group, SUM(id);
{code}
                
> Schema errors when using PigStorage and none when using BinStorage in FOREACH??
> -------------------------------------------------------------------------------
>
>                 Key: PIG-798
>                 URL: https://issues.apache.org/jira/browse/PIG-798
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.0
>            Reporter: Viraj Bhat
>            Assignee: Alan Gates
>         Attachments: binstoragecreateop, schemaerr.pig, visits.txt
>
>
> In the following script I have a tab separated text file, which I load using PigStorage()
and store using BinStorage()
> {code}
> A = load '/user/viraj/visits.txt' using PigStorage() as (name:chararray, url:chararray,
time:chararray);
> B = group A by name;
> store B into '/user/viraj/binstoragecreateop' using BinStorage();
> dump B;
> {code}
> I later load file 'binstoragecreateop' in the following way.
> {code}
> A = load '/user/viraj/binstoragecreateop' using BinStorage();
> B = foreach A generate $0 as name:chararray;
> dump B;
> {code}
> Result
> =======================================================================
> (Amy)
> (Fred)
> =======================================================================
> The above code work properly and returns the right results. If I use PigStorage() to
achieve the same, I get the following error.
> {code}
> A = load '/user/viraj/visits.txt' using PigStorage();
> B = foreach A generate $0 as name:chararray;
> dump B;
> {code}
> =======================================================================
> {code}
> 2009-05-02 03:58:50,662 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1022: Type
mismatch merging schema prefix. Field Schema: bytearray. Other Field Schema: name: chararray
> Details at logfile: /home/viraj/pig-svn/trunk/pig_1241236728311.log
> {code}
> =======================================================================
> So why should the semantics of BinStorage() be different from PigStorage() where is ok
not to specify a schema??? Should it not be consistent across both.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message