hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy V. Ryaboy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1333) API interface to Pig
Date Mon, 14 Jun 2010 20:31:15 GMT

    [ https://issues.apache.org/jira/browse/PIG-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878732#action_12878732
] 

Dmitriy V. Ryaboy commented on PIG-1333:
----------------------------------------

bq. I'm not sure we should make all Hadoop counters available through the new API. How useful
will it be to the users? I'm open to suggestions. 

Can't speak for other users, but we use counters quite a bit with Elephant Bird and some internal
code for keeping track of timed out service requests, unparsable records, and more. The @MonitoredUDF
annotation I proposed in PIG-1427 uses counters to report on runaway udfs that get killed.

I think the question isn't so much why would you expose them, as why wouldn't you expose them...

> API interface to Pig
> --------------------
>
>                 Key: PIG-1333
>                 URL: https://issues.apache.org/jira/browse/PIG-1333
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Olga Natkovich
>            Assignee: Richard Ding
>             Fix For: 0.8.0
>
>         Attachments: PIG-1333.patch, PIG-1333_1.patch
>
>
> It would be nice to make Pig more friendly for applications like workflow that would
be executing pig scripts on user behalf.
> Currently, they would have to use pig command line to execute the code; however, this
has limitation on the kind of output that would be delivered. For instance, it is hard to
produce error information that is easy to use programatically or collect statistics.
> The proposal is to create a class that mimics the behavior of the Main but gives users
a status object back. The the main code of pig would look somethig like:
> public static void main(String args[])
> {
>     PigStatus ps = PigMain.exec(args);
>     exit (PigStatus.rc);
> }
> We need to define the following:
> - Content of PigStatus. It should at least include
>    * return code
>    * error string
>    * exception 
>    * statistics
> - A way to propagate the status class through pig code

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message