pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1717) pig needs to call setPartitionFilter if schema is null but getPartitionKeys is not
Date Fri, 21 Jan 2011 19:53:45 GMT

    [ https://issues.apache.org/jira/browse/PIG-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12984897#action_12984897

Daniel Dai commented on PIG-1717:

Configuration in LOLoad should not be null in regular path. However, in the test, we may pass
null for convenience. You can change the test to pass the right configuration to LOLoad construct
in those tests.

> pig needs to call setPartitionFilter if schema is null but getPartitionKeys is not
> ----------------------------------------------------------------------------------
>                 Key: PIG-1717
>                 URL: https://issues.apache.org/jira/browse/PIG-1717
>             Project: Pig
>          Issue Type: Improvement
>          Components: impl
>    Affects Versions: 0.9.0
>            Reporter: Gerrit Jansen van Vuuren
>            Assignee: Gerrit Jansen van Vuuren
>            Priority: Minor
>             Fix For: 0.9.0
>         Attachments: patchReleaseAuditWarnings.txt.gz, PIG-1717.patch, PIG-1717.v1.patch,
testlog.tgz, trunkReleaseAuditWarnings.txt.gz
> I'm writing a loader that works with hive style partitioning e.g. /logs/type1/daydate=2010-11-01
> The loader does not know the schema upfront and this is something that the user adds
in the script using the AS clause.
> The problem is that this user defined schema is not available to the loader, so the loader
cannot return any schema, the Loader does know what the partition keys are and pig needs in
some way to know about these partition keys. 
> Currently if the schema is null pig never calls the LoadMetaData:getPartitionKeys method
or the setPartitionFilter method.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message