pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher Olston (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-572) A PigServer.registerScript() method, which lets a client programmatically register a Pig Script.
Date Mon, 05 Jan 2009 18:33:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660843#action_12660843
] 

Christopher Olston commented on PIG-572:
----------------------------------------

Alan asked about use cases. This feature allows users to mix pre-existing scripts with new
commands. Here's the main use case I envision:

There's a shared data repository with a complex schema and rich data (e.g. web pages and usage
logs). A set of users wish to perform a particular type of analysis (e.g., click rates over
time). User A writes a script that extracts a handful of fields relevant to this class of
analysis, transforms them into a form that facilitates the analysis, and exports them in a
flattened schema. User B wants to perform this type of analysis, building off User A's script
and then issuing follow-on commands to the PigServer.

Effectively, it gives us a (virtual, not materialized) view mechanism. Views are known to
be extremely useful in conventional database management systems. They may be even more useful
in Pig, where data is sometimes stored in complex ways (nested schemas) but for some purposes
it's easier to deal with normalized relational data.


> A PigServer.registerScript() method, which lets a client programmatically register a
Pig Script.
> ------------------------------------------------------------------------------------------------
>
>                 Key: PIG-572
>                 URL: https://issues.apache.org/jira/browse/PIG-572
>             Project: Pig
>          Issue Type: New Feature
>    Affects Versions: types_branch
>            Reporter: Shubham Chopra
>            Priority: Minor
>             Fix For: types_branch
>
>         Attachments: registerScript.patch
>
>
> A PigServer.registerScript() method, which lets a client programmatically register a
Pig Script.
> For example, say theres a script my_script.pig with the following content:
> a = load '/data/my_data.txt';
> b = filter a by $0 > '0';
> The function lets you use something like the following:
> pigServer.registerScript("my_script.pig");
> pigServer.registerQuery("c = foreach b generate $2, $3;");
> pigServer.store("c");

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message