hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sam Pullara (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-357) progress reported on every tuple
Date Mon, 04 Aug 2008 18:34:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619634#action_12619634

Sam Pullara commented on PIG-357:

I would do this using a timer rather than a absolute number of tuples due to the vagaries
of how long processing might take.  Maybe every 10s?  You could check a boolean to see if
the timer went off every tuple, if so report and reset timer.

> progress reported on every tuple
> --------------------------------
>                 Key: PIG-357
>                 URL: https://issues.apache.org/jira/browse/PIG-357
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: types_branch
>            Reporter: Olga Natkovich
>             Fix For: types_branch
> Currently, if the reporter is set, we report progress on every tuple. This could be too
expensive and impact performance. In the old code, we used to do it on every 1000th tuple
or something like that.
> We might want to go to similar model.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message