hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1460) UDF manual and javadocs should make clear how to use RequiredFieldList
Date Fri, 18 Jun 2010 19:48:23 GMT

    [ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880299#action_12880299

Alan Gates commented on PIG-1460:

>From email thread on the pig-user list:

The documentation is also poor when it comes to describing what the
RequiredFieldList even is.

It has a name and an index field.   The code itself seems to allow for
either of these to be filled.  What do they mean?

Is it:
the schema returned by the loader is:
(id: int, name: chararray, department: chararray)

The RequiredFieldList is [ ("department", 1) , ("id", 0) ]

What does that mean?
* The name is the field name requested, and the index is the location it
should be in the result?  so return (id: int, department: chararray)?
* The index is the index in the source schema, and the name is for
renaming, so return (department: chararray, id: int) (where the data in
department is actualy that from the original's name field)?
* The location in the RequiredFieldList array is the 'destination'
requested, the name is optional (if the schema had one) and the index is the
location in the original schema.  so the above RequiredFieldList is actually
impossible, since "department" is always index 2.

The last is the correct answer.

> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>                 Key: PIG-1460
>                 URL: https://issues.apache.org/jira/browse/PIG-1460
>             Project: Pig
>          Issue Type: Bug
>          Components: documentation
>    Affects Versions: 0.7.0
>            Reporter: Alan Gates
>            Priority: Minor
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed
to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the
contents of that list.  The javadoc is similarly vague. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message