hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miles Osborne" <mi...@inf.ed.ac.uk>
Subject Re: Question regarding reduce tasks
Date Mon, 03 Nov 2008 16:28:54 GMT
you can't guarantee that a reducer (or mapper for that matter) will be
executed exactly once unless you turn-off preemptive scheduling.  but,
a distinct key gets sent to a single reducer, so yes, only one reducer
will see a particulat key + associated values


2008/11/3 Ryan LeCompte <lecompte@gmail.com>:
> Hello,
> Is it safe to assume that only one reduce task will ever operate on
> values for a particular key? Or is it possible that more than one
> reduce task can work on values for the same key? The reason I ask is
> because I want to ensure that a piece of code that I write at the end
> of my reducer method will only ever be executed once after all values
> for a particular key are aggregated/summed.
> Thanks,
> Ryan

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

View raw message