hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: Can caller mutate value after calling OutputCollector.collect(key,value) ?
Date Mon, 13 Feb 2012 00:15:12 GMT
Hi

On Sun, Feb 12, 2012 at 8:58 PM, Mike Spreitzer <mspreitz@us.ibm.com> wrote:
> I have a question about the contract in the
> org.apache.hadoop.mapred.OutputCollector interface.  If it matters, let us
> say we are talking about Hadoop-1.0.0.  In my map or reduce method, after it
> calls output.collect(key,value), is it allowed to mutate (side effect)
> either the key or the value?
>
> If the answer is "it depends on the OutputFormat", then what is the answer
> for the more prominent ones, such as SequenceFileFormat and
> TextOutputFormat?

In both cases the key is serialized immediately so the caller cannot
mutate the key after calling collect.

http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0/src/mapred/org/apache/hadoop/mapred/TextOutputFormat.java
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0/src/mapred/org/apache/hadoop/mapred/SequenceFileOutputFormat.java
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0/src/core/org/apache/hadoop/io/SequenceFile.java

-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Mime
View raw message