hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J" <ha...@cloudera.com>
Subject Review Request: MAPREDUCE-1347. Missing synchronization in MultipleOutputFormat.
Date Fri, 24 Jun 2011 19:06:46 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/953/
-----------------------------------------------------------

Review request for hadoop-mapreduce and Todd Lipcon.


Summary
-------

Used the makeComputingMap from Guava's MapMaker to provide a thread safe way of creating a
RecordWriter cache.

For some reason, the map is not really caching it and is instead trying to apply() over and
over again for the same key-value pairs.


This addresses bug MAPREDUCE-1347.
    http://issues.apache.org/jira/browse/MAPREDUCE-1347


Diffs
-----

  mapreduce/ivy.xml 85ee014 
  mapreduce/ivy/libraries.properties 9d40aaa 
  mapreduce/src/java/org/apache/hadoop/mapred/lib/MultipleOutputFormat.java b8944f1 
  mapreduce/src/test/mapred/org/apache/hadoop/mapred/TestMultipleTextOutputFormat.java 14c097d


Diff: https://reviews.apache.org/r/953/diff


Testing
-------

Added a test case, but it fails with the current behavior of MapMaker's makeComputingMap()
(would pass if its alright)


Thanks,

Harsh


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message