hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-335) Casting does not work in certain cases with multiple loads
Date Thu, 02 Oct 2008 18:49:46 GMT

    [ https://issues.apache.org/jira/browse/PIG-335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636426#action_12636426
] 

Santhosh Srinivasan commented on PIG-335:
-----------------------------------------

1. Added documentation to Schema.java describing the use of the canonical maps

2. This is a bug. It requires a change in the front end design to handle multi-column projects.
I will open a new JIRA for this.

3. Changed the error message to "Cannot resolve load function to use for casting from " +
DataType.findTypeName(inputType) + " to  ...

4. The error message in the getLoadFunc message is appropriate, the correction I made here
was to drop the word interface.

5. For Unions, Cross and Cogroup, depending on the column accessed, it will either result
in the  appropriate traversal or a branch and unbound find.

> Casting does not work in certain cases with multiple loads
> ----------------------------------------------------------
>
>                 Key: PIG-335
>                 URL: https://issues.apache.org/jira/browse/PIG-335
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: types_branch
>            Reporter: Alan Gates
>            Assignee: Santhosh Srinivasan
>            Priority: Critical
>             Fix For: types_branch
>
>         Attachments: PIG_335.patch, PIG_335_1.patch
>
>
> Given a script like:
> A = load 'bla' as (x, y) using Loader1();
> B = load 'morebla' as (s, t) using Loader2();
> C = cogroup A by x, B by s;
> D = foreach C generate flatten(A), flatten(B);
> E = foreach D generate x, y, t + 1;
> In this case, in the last foreach, a cast will need to be added to t + 1 to allow t (a
byte array) to be added to an integer.  We use load functions to handle this late casting.
 The issue is that we do not currently have a way to know whether to use Loader1 or Loader2
to cast the data.  We need to track the lineage of fields so that the cast operator can select
the correct loader.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message