hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1428) Add getPigStatusReporter() to PigHadoopLogger
Date Thu, 10 Jun 2010 22:02:19 GMT

    [ https://issues.apache.org/jira/browse/PIG-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877591#action_12877591
] 

Ashutosh Chauhan commented on PIG-1428:
---------------------------------------

So, I read through PIG-889. It seems that there never was a documented way to use counters,
reporters etc from UDFs, Load/Store Funcs. Actually, there is a hacky way to do it, which
exists in DefaultAbstractBag.java 
{code}
    protected void incSpillCount(Enum counter) {
        // Increment the spill count
        // warn is a misnomer. The function updates the counter. If the update
        // fails, it dumps a warning
        PigHadoopLogger.getInstance().warn(this, "Spill counter incremented", counter);
    }
{code}
But in PIG-889 Santhosh has argued against for this (mis)use of PigLogger. I think we need
to provide a formal way to Pig users to access counters, reporters from our interfaces (UDFs,
L/S) as PigHadoopLogger is designed for error-handling (warning aggregation in particular)
and not for this purpose. And we shall mark this class as Internal only, before some one starts
using it. With the same argument, above method where Pig is internally making use of its own
Counters is flawed and needs to be corrected.

> Add getPigStatusReporter() to PigHadoopLogger
> ---------------------------------------------
>
>                 Key: PIG-1428
>                 URL: https://issues.apache.org/jira/browse/PIG-1428
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.7.0
>            Reporter: Ashutosh Chauhan
>            Assignee: Dmitriy V. Ryaboy
>             Fix For: 0.8.0
>
>         Attachments: PIG-1428.patch, PIG-1428.patch
>
>
> Without this getter method, its not possible to get counters, report progress etc. from
UDFs. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message