hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stan Rosenberg <srosenb...@proclivitysystems.com>
Subject Re: race condition in hadoop 0.20.2 (cdh3u1)
Date Tue, 17 Jan 2012 23:51:02 GMT
On Tue, Jan 17, 2012 at 6:38 PM, Brock Noland <brock@cloudera.com> wrote:
> This class is invalid. A single thread will be executing your mapper
> or reducer but there will be multiple threads (background threads such
> as the SpillThread) creating MyKey instances which is exactly what you
> are seeing. This is by design.
>

Could you please refer me to where this design decision/assumption
is/was documented?  Imho, this assumption clashes with the overall
object re-use methodology. I would have at least considered
making 'readFields' and 'write' synchronized, even if it is to
indicate that there are multiple threads executing
serialization/de-serialization.  (As only a few threads are competing
in this case, the performance penalty would have been negligible.)

Thanks,

stan

Mime
View raw message