hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-966) Proposed rework for LoadFunc, StoreFunc, and Slice/r interfaces
Date Wed, 02 Dec 2009 17:52:20 GMT

    [ https://issues.apache.org/jira/browse/PIG-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784931#action_12784931
] 

Alan Gates commented on PIG-966:
--------------------------------

You can make an argument for putting it in either place.  I argue for putting it in for a
couple of reasons:

It is useful to a large number of potential optimizations.

Unlike most other statistics, it can be used in correctness checks (eg the user asked for
a merge join, is the data sorted on the join key?)

The only downside I can see is that some systems that will understand column names and types
won't necessarily understand sortedness (like json).  But it's no harder for the loader to
figure out sortedness for the schema than it is for the statistics.

> Proposed rework for LoadFunc, StoreFunc, and Slice/r interfaces
> ---------------------------------------------------------------
>
>                 Key: PIG-966
>                 URL: https://issues.apache.org/jira/browse/PIG-966
>             Project: Pig
>          Issue Type: Improvement
>          Components: impl
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>
> I propose that we rework the LoadFunc, StoreFunc, and Slice/r interfaces significantly.
 See http://wiki.apache.org/pig/LoadStoreRedesignProposal for full details

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message