[ https://issues.apache.org/jira/browse/PIG-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14592881#comment-14592881
]
Michael Howard commented on PIG-4609:
-------------------------------------
I am also interested in using counters in pig.
I posted the message below to user@pig.apache.org on 2015-06-18.
Thus far I have only received one response, saying that the PigStatusReporter does not asynchronously
update the counters.
----
accessing custom counters during execution of pig job
----
My questions relate to custom "counters" in order to dynamically monitor progress during execution
of a pig job.
I would like to increment some custom counters in the context of pig UDFs.
I would like to access these counters during execution under differnt ExecTypes ... local/mapreduce/tez
I see that PigStatusReporter.getInstance().incrCounter(...) is the recommended way to increment
counters so that they are accessible under both MR and tez execution models.
I do not see how one can fetch the current value of a counter during execution via PigRunner
or PigServer.
I expected to find access to counters under PigStats, but have been unable to locate a way
to access counters.
Q: How does one retrieve current values of counters during execution of a pig script?
Thanks,
Michael
> PIG : How to use counter in pig script
> --------------------------------------
>
> Key: PIG-4609
> URL: https://issues.apache.org/jira/browse/PIG-4609
> Project: Pig
> Issue Type: Task
> Components: internal-udfs, parser, piggybank
> Affects Versions: 0.8.0, 0.11
> Reporter: Sudhir Ranjan
>
> Hi There,
> How can I use counter to count the total #of records processed through pig script.
> I know that it can be achieved by writing & invoking a UDF from pig script or by
using GROUPBY & COUNT(), but here I don't want to go this way.
> I need the way in PIG like the way we are accessing the counter as
> context.getCounter(MyCounters.Counter).increment(1);
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
|