hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Lewis <lordjoe2...@gmail.com>
Subject What is the cost of using a counter
Date Thu, 08 Dec 2011 18:48:41 GMT
I have several counters that I maintain to allow me to keep statistics on
critical operations. I have my code incrementing the counters in an inner
loop
partly to make sure my job is not killed for not making progress. It would
be very easy to keep an internal counter and increment the Hadoop value
less frequently.  Assuming
I am currently incrementing a counter several million times in a reduce
task- is this costing me performance and would I be better off incrementing
less frequently

-- 
Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033
206-384-1340 (cell)
Skype lordjoe_com

Mime
View raw message