hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From d...@apache.org
Subject svn commit: r639223 - in /hadoop/core/trunk: CHANGES.txt docs/changes.html docs/mapred_tutorial.html docs/mapred_tutorial.pdf src/docs/src/documentation/content/xdocs/mapred_tutorial.xml src/java/org/apache/hadoop/mapred/JobConf.java
Date Thu, 20 Mar 2008 09:31:38 GMT
Author: ddas
Date: Thu Mar 20 02:31:37 2008
New Revision: 639223

URL: http://svn.apache.org/viewvc?rev=639223&view=rev
Log:
HADOOP-3042. Updates the Javadoc in JobConf.getOutputPath to reflect the actual temporary
path. Contributed by Amareshwari Sriramadasu.

Modified:
    hadoop/core/trunk/CHANGES.txt
    hadoop/core/trunk/docs/changes.html
    hadoop/core/trunk/docs/mapred_tutorial.html
    hadoop/core/trunk/docs/mapred_tutorial.pdf
    hadoop/core/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
    hadoop/core/trunk/src/java/org/apache/hadoop/mapred/JobConf.java

Modified: hadoop/core/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/CHANGES.txt?rev=639223&r1=639222&r2=639223&view=diff
==============================================================================
--- hadoop/core/trunk/CHANGES.txt (original)
+++ hadoop/core/trunk/CHANGES.txt Thu Mar 20 02:31:37 2008
@@ -332,6 +332,9 @@
     HADOOP-3003. FileSystem cache key is updated after a 
     FileSystem object is created. (Tsz Wo (Nicholas), SZE via dhruba)
 
+    HADOOP-3042. Updates the Javadoc in JobConf.getOutputPath to reflect 
+    the actual temporary path. (Amareshwari Sriramadasu via ddas)
+
 Release 0.16.1 - 2008-03-13
 
   INCOMPATIBLE CHANGES

Modified: hadoop/core/trunk/docs/changes.html
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/docs/changes.html?rev=639223&r1=639222&r2=639223&view=diff
==============================================================================
--- hadoop/core/trunk/docs/changes.html (original)
+++ hadoop/core/trunk/docs/changes.html Thu Mar 20 02:31:37 2008
@@ -36,7 +36,7 @@
     function collapse() {
       for (var i = 0; i < document.getElementsByTagName("ul").length; i++) {
         var list = document.getElementsByTagName("ul")[i];
-        if (list.id != 'trunk_(unreleased_changes)_' && list.id != 'release_0.16.1_-_2008-03-13_')
{
+        if (list.id != 'trunk_(unreleased_changes)_' && list.id != 'release_0.16.2_-_unreleased_')
{
           list.style.display = "none";
         }
       }
@@ -56,7 +56,7 @@
 </a></h2>
 <ul id="trunk_(unreleased_changes)_">
   <li><a href="javascript:toggleList('trunk_(unreleased_changes)_._incompatible_changes_')">
 INCOMPATIBLE CHANGES
-</a>&nbsp;&nbsp;&nbsp;(9)
+</a>&nbsp;&nbsp;&nbsp;(10)
     <ol id="trunk_(unreleased_changes)_._incompatible_changes_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2786">HADOOP-2786</a>.
 Move hbase out of hadoop core
 </li>
@@ -77,10 +77,12 @@
 and isDir(String) from ClientProtocol. ClientProtocol version changed
 from 26 to 27. (Tsz Wo (Nicholas), SZE via cdouglas)
 </li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2822">HADOOP-2822</a>.
Remove depreceted code for classes InputFormatBase and
+PhasedFileSystem.<br />(Amareshwari Sriramadasu via enis)</li>
     </ol>
   </li>
   <li><a href="javascript:toggleList('trunk_(unreleased_changes)_._new_features_')">
 NEW FEATURES
-</a>&nbsp;&nbsp;&nbsp;(6)
+</a>&nbsp;&nbsp;&nbsp;(7)
     <ol id="trunk_(unreleased_changes)_._new_features_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-1398">HADOOP-1398</a>.
 Add HBase in-memory block cache.<br />(tomwhite)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2178">HADOOP-2178</a>.
 Job History on DFS.<br />(Amareshwari Sri Ramadasu via ddas)</li>
@@ -94,10 +96,12 @@
 config params to map records to different output files.<br />(Runping Qi via cdouglas)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2346">HADOOP-2346</a>.
Utilities to support timeout while writing to sockets.
 DFSClient and DataNode sockets have 10min write timeout.<br />(rangadi)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2951">HADOOP-2951</a>.
 Add a contrib module that provides a utility to
+build or update Lucene indexes using Map/Reduce.<br />(Ning Li via cutting)</li>
     </ol>
   </li>
   <li><a href="javascript:toggleList('trunk_(unreleased_changes)_._improvements_')">
 IMPROVEMENTS
-</a>&nbsp;&nbsp;&nbsp;(14)
+</a>&nbsp;&nbsp;&nbsp;(20)
     <ol id="trunk_(unreleased_changes)_._improvements_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2655">HADOOP-2655</a>.
Copy on write for data and metadata files in the
 presence of snapshots. Needed for supporting appends to HDFS
@@ -127,10 +131,22 @@
 </li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2804">HADOOP-2804</a>.
 Add support to publish CHANGES.txt as HTML when running
 the Ant 'docs' target.<br />(nigel)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2559">HADOOP-2559</a>.
Change DFS block placement to allocate the first replica
+locally, the second off-rack, and the third intra-rack from the
+second.<br />(lohit vijayarenu via cdouglas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2939">HADOOP-2939</a>.
Make the automated patch testing process an executable
+Ant target, test-patch.<br />(nigel)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2239">HADOOP-2239</a>.
Add HsftpFileSystem to permit transferring files over ssl.<br />(cdouglas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2910">HADOOP-2910</a>.
Throttle IPC Client/Server during bursts of
+requests or server slowdown.<br />(Hairong Kuang via dhruba)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2848">HADOOP-2848</a>.
[HOD]hod -o list and deallocate works even after deleting
+the cluster directory.<br />(Hemanth Yamijala via ddas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2899">HADOOP-2899</a>.
[HOD] Cleans up hdfs:///mapredsystem directory after
+deallocation.<br />(Hemanth Yamijala via ddas)</li>
     </ol>
   </li>
   <li><a href="javascript:toggleList('trunk_(unreleased_changes)_._optimizations_')">
 OPTIMIZATIONS
-</a>&nbsp;&nbsp;&nbsp;(4)
+</a>&nbsp;&nbsp;&nbsp;(7)
     <ol id="trunk_(unreleased_changes)_._optimizations_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2790">HADOOP-2790</a>.
 Fixed inefficient method hasSpeculativeTask by removing
 repetitive calls to get the current time and late checking to see if
@@ -143,10 +159,16 @@
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2423">HADOOP-2423</a>.
 Code optimization in FSNamesystem.mkdirs.
 (Tsz Wo (Nicholas), SZE via dhruba)
 </li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2606">HADOOP-2606</a>.
ReplicationMonitor selects data-nodes to replicate directly
+from needed replication blocks instead of looking up for the blocks for
+each live data-node.<br />(shv)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2148">HADOOP-2148</a>.
Eliminate redundant data-node blockMap lookups.<br />(shv)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2027">HADOOP-2027</a>.
Return the number of bytes in each block in a file
+via a single rpc to the namenode to speed up job planning.<br />(Lohit Vijaya Renu
via omalley)</li>
     </ol>
   </li>
   <li><a href="javascript:toggleList('trunk_(unreleased_changes)_._bug_fixes_')">
 BUG FIXES
-</a>&nbsp;&nbsp;&nbsp;(42)
+</a>&nbsp;&nbsp;&nbsp;(51)
     <ol id="trunk_(unreleased_changes)_._bug_fixes_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2195">HADOOP-2195</a>.
'-mkdir' behaviour is now closer to Linux shell in case of
 errors.<br />(Mahadev Konar via rangadi)</li>
@@ -227,12 +249,52 @@
 with different sizes to the namenode, the namenode picks the
 replica(s) with the largest size as the only valid replica(s).<br />(dhruba)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2825">HADOOP-2825</a>.
Deprecated MapOutputLocation.getFile() is removed.<br />(Amareshwari Sri Ramadasu via
ddas)</li>
-      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2806">HADOOP-2806</a>.
Fixes a streaming document.<br />(Amareshwari Sri Ramadasu via ddas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2806">HADOOP-2806</a>.
Fixes a streaming document.<br />(Amareshwari Sriramadasu via ddas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2924">HADOOP-2924</a>.
Fixes an address problem to do with TaskTracker binding
+to an address.<br />(Vinod Kumar Vavilapalli via ddas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2970">HADOOP-2970</a>.
Fixes a problem to do with Wrong class definition for
+hodlib/Hod/hod.py for Python &lt; 2.5.1.<br />(Vinod Kumar Vavilapalli via ddas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-3008">HADOOP-3008</a>.
SocketIOWithTimeout throws InterruptedIOException if the
+thread is interrupted while it is waiting.<br />(rangadi)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-3006">HADOOP-3006</a>.
Fix wrong packet size reported by DataNode when a block
+is being replicated.<br />(rangadi)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-3029">HADOOP-3029</a>.
Datanode prints log message "firstbadlink" only if
+it detects a bad connection to another datanode in the pipeline.<br />(dhruba)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-3030">HADOOP-3030</a>.
Release reserved space for file in InMemoryFileSystem if
+checksum reservation fails.<br />(Devaraj Das via cdouglas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-3036">HADOOP-3036</a>.
Fix findbugs warnings in UpgradeUtilities.<br />(Konstantin
+Shvachko via cdouglas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2783">HADOOP-2783</a>.
Fixes a problem to do with import in
+hod/hodlib/Common/xmlrpc.py.<br />(Vinod Kumar Vavilapalli via ddas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2936">HADOOP-2936</a>.
Fixes HOD in a way that it generates hdfs://host:port on the
+client side configs.<br />(Vinod Kumar Vavilapalli via ddas)</li>
     </ol>
   </li>
 </ul>
-<h2><a href="javascript:toggleList('release_0.16.1_-_2008-03-13_')">Release 0.16.1
- 2008-03-13
+<h2><a href="javascript:toggleList('release_0.16.2_-_unreleased_')">Release 0.16.2
- Unreleased
 </a></h2>
+<ul id="release_0.16.2_-_unreleased_">
+  <li><a href="javascript:toggleList('release_0.16.2_-_unreleased_._bug_fixes_')">
 BUG FIXES
+</a>&nbsp;&nbsp;&nbsp;(5)
+    <ol id="release_0.16.2_-_unreleased_._bug_fixes_">
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-3011">HADOOP-3011</a>.
Prohibit distcp from overwriting directories on the
+destination filesystem with files.<br />(cdouglas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-3033">HADOOP-3033</a>.
The BlockReceiver thread in the datanode writes data to
+the block file, changes file position (if needed) and flushes all by
+itself. The PacketResponder thread does not flush block file.<br />(dhruba)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2978">HADOOP-2978</a>.
Fixes the JobHistory log format for counters.<br />(Runping Qi via ddas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2985">HADOOP-2985</a>.
Fixes LocalJobRunner to tolerate null job output path.
+Also makes the _temporary a constant in MRConstants.java.<br />(Amareshwari Sriramadasu
via ddas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-3003">HADOOP-3003</a>.
FileSystem cache key is updated after a
+FileSystem object is created. (Tsz Wo (Nicholas), SZE via dhruba)
+</li>
+    </ol>
+  </li>
+</ul>
+<h2><a href="javascript:toggleList('older')">Older Releases</a></h2>
+<ul id="older">
+<h3><a href="javascript:toggleList('release_0.16.1_-_2008-03-13_')">Release 0.16.1
- 2008-03-13
+</a></h3>
 <ul id="release_0.16.1_-_2008-03-13_">
   <li><a href="javascript:toggleList('release_0.16.1_-_2008-03-13_._incompatible_changes_')">
 INCOMPATIBLE CHANGES
 </a>&nbsp;&nbsp;&nbsp;(2)
@@ -337,8 +399,6 @@
     </ol>
   </li>
 </ul>
-<h2><a href="javascript:toggleList('older')">Older Releases</a></h2>
-<ul id="older">
 <h3><a href="javascript:toggleList('release_0.16.0_-_2008-02-07_')">Release 0.16.0
- 2008-02-07
 </a></h3>
 <ul id="release_0.16.0_-_2008-02-07_">

Modified: hadoop/core/trunk/docs/mapred_tutorial.html
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/docs/mapred_tutorial.html?rev=639223&r1=639222&r2=639223&view=diff
==============================================================================
--- hadoop/core/trunk/docs/mapred_tutorial.html (original)
+++ hadoop/core/trunk/docs/mapred_tutorial.html Thu Mar 20 02:31:37 2008
@@ -1751,10 +1751,11 @@
           (using the taskid, say <span class="codefrag">task_200709221812_0001_m_000000_0</span>),

           not just per task.</p>
 <p>To avoid these issues the Map-Reduce framework maintains a special 
-          <span class="codefrag">${mapred.output.dir}/_${taskid}</span> sub-directory
for each 
-          task-attempt on the <span class="codefrag">FileSystem</span> where
the output of the 
-          task-attempt is stored. On successful completion of the task-attempt, 
-          the files in the <span class="codefrag">${mapred.output.dir}/_${taskid}</span>
(only) 
+          <span class="codefrag">${mapred.output.dir}/_temporary/_${taskid}</span>
sub-directory
+          for each task-attempt on the <span class="codefrag">FileSystem</span>
where the output
+          of the task-attempt is stored. On successful completion of the 
+          task-attempt, the files in the 
+          <span class="codefrag">${mapred.output.dir}/_temporary/_${taskid}</span>
(only) 
           are <em>promoted</em> to <span class="codefrag">${mapred.output.dir}</span>.
Of course, 
           the framework discards the sub-directory of unsuccessful task-attempts. 
           This process is completely transparent to the application.</p>

Modified: hadoop/core/trunk/docs/mapred_tutorial.pdf
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/docs/mapred_tutorial.pdf?rev=639223&r1=639222&r2=639223&view=diff
==============================================================================
--- hadoop/core/trunk/docs/mapred_tutorial.pdf (original)
+++ hadoop/core/trunk/docs/mapred_tutorial.pdf Thu Mar 20 02:31:37 2008
@@ -1383,10 +1383,10 @@
 >>
 endobj
 161 0 obj
-<< /Length 2455 /Filter [ /ASCII85Decode /FlateDecode ]
+<< /Length 2462 /Filter [ /ASCII85Decode /FlateDecode ]
  >>
 stream
-Gat=-gQ(#H&:O:SkTN38JX@%K4'E>3K&*4'MPOCaIHVFSg7iC0ZD?]iNY1@03truV-qd_k/Xf/7LZAliS2iBS\Q[di(7uDmfYW-A(P,.O_f]8[drsg0Q]>Vhb70Cs)!f/5MjI3XfX=3MgI;B#DQ,o>I("nJAn,Jj^XQ)gN_?<h)lp[BmhI/#`dc"n/XSLFPO@bdo\8iHB)DPAGnPnF'%b-d27m3q47nWb<F/bM1)HE>MDr2*ZP7g+9pg@Qej&n);-*<"iX=Z,M0Rmlg$HZ>'4-JrZ`!&[_mg8I.P$O]cN?BTN-=AsR#*Ts$/+mF&3N)7K>hEP7us61:Bd3k`9W$"/dSms'%S)_I??\4i@rjM$GlNFY![fVe^sR=2H^Xk;oFqLAp.gI6`ctu/0hnGBf3E@\eH;K<7_*J+SWG92PN/Cj^m?mitk0J@WlC42nqC,d#',W!.,8%htTlqRH5(u&l%-U(,i`cQBg+KK\)s\\L!ZsWWA.o"N9cC]b4[%jlC8Zlm:el0,)4?FR-H,Li+Ub?749nf:GEE0Nde\P?:RFS=sfg[l<9T#LW*jokbfn,$Cp-n;<P\qsPXlQ8_YDH/66J<Hm>:EmC59CCud+*IBP#+Ki_TkAD\0)nou?FfO=2ABJ_L!m:*sBpl0CXHi2E/;7=qpAgeXCIuKE<c*A5E*DcgI"=:uR!$KWPjrI.!M/>VM`3^kM\*X4OCH@Db9_>pE`?TH#nZW$<V&UOg#ZnY%hi+5663p4"'aeqDKmUP']Y_?A9EL@[mhIYAQ`Nd6X6<rX$_W0TWECL\VQSE^7SnFgB>$R^@'f!J<!bR[Xj54TiG.;mGn%;C2"G*6s+A>2c>bsiGE4+e7fi!>E#+90EY*><gO,mI.pE:BNb?#"B+/T%+'HQNF!;hD*&0\//RO]iOuFVR\R;%60]Fi(Pd!eIEQoD_oK]LDnc3+dj0M-lC2'Y*gk!n/J70Z&V/!&h!=cJH"bBKO7WQdjc2Q
 #6nOJ)U8l];F$1R6cfll4KeT5`ammCAZBr3*YO`)+[k@>(h[KPuLDWMYpf)sS?.K[R>i5W8_(u25W))*!Bq"[&0E2:W4IC2DMKr#%[,p_Zd94m+q=1^BWu8rJ8^=bYT:RPIHiBDOTb`f3L:agK!LkW?VB\&:E%QjH5khhP0h=9s2GN;kA;qm\57qVlpJg\Ll^m8^Tnjn<!lFfgMhbqPeJ3P&'gaND'5b"[?8%Er+7B7jXp!eK\i`iT6e)+\/=$'X@!BtUd&c)p/0s9D@)\uG.jBXjHBj/8<CSi<Ff@83ESc,'i6nEL8l$Moq8CYc.QbA7.rCckY1i;.N[btNUGH$>*WV<B0p=OZ5eDb4%LNRLpd>*B#bJ,\U21_\ck4?b9='i+3o-SlPZ[ELc!nog\T/9AkA+II6l65Q;2WKm4(jI*/iUEKLR`FG*6U^$@j6aak3I*tW>K1mQMEV4!,r\$E]cEeYR@O0We.-[%QCNsm\bbjjW[-H9'[n'>IN3`_<TIC]\thCpa9kn5'!=]XVd+,V8X^3VP(rqZUXpQ/V@)>H&=fRgOfE1+[4LtEItgjW@X+$V_ZGnnI:Z0Z<Z5FHgqQ9f%CsM?"j4n0-E$Jk=]'8=TA"1F5.\6*K:T5NJ/,"9p9HLeb/8J:$qX-&d.RVnVi*?Vg?>5\C_^JJNGH8cMf_+^WOu\Cdm*q^R$/98s3+o&j2pNm>:2%j(fXL+K.p+.#j:`\1S.nRaY/EO.ik?'4)q_V-,,;FA?W$;^8r3"BJ8&.%Nt''<.no7X87ZSj%k$?QrUES52\d:9!!o-Q"&1Ob)9_%"c$/?&j9#i.C;8c?g-WPmJ^imfFmd["u"jBO@9\&nf]"5h0=?KhQ]26AY/&*FEZ=9H$Nm;PfrPRAp-Y+ri92b9q:o5]KM`UHGNpYB)JIJf"7r3KK)=*#=o\a2SaejN^j@hN8A?C61=p`<1j_.Hnfg4Zps,WTOGLj]Nku[8UBW@Vsj;g1b
 f@7ds0*J->:S*p6t2+ic;8j'SY0R,D#qW9I_1%H)W2Fo(J-Lu2^r1i=KBWLOP)Zp,4q&\cEQ'Wu1_><)emBXb*T')h!WM4T8V,="b93nRtB*l@KNDH-rjKV^"g[RmpiRWqmA,N]D''W7>k7'$V7fDZlV&[Tm\+)U.icC#Tf6LCXac)1/aDZ\d3IAo,HEtZ9?ra9E:.Auil88<7lFP=eO+^k/X2ebE:]XKG$%rJ=V/k-HIKGX_a>/bMC2ETCZ!7B.oI*X5^djNGf#.H_=J8X<SdN2p]AV'^S)]iTo/e@8nP4[E=5H!,6)MEQ(2J=Vl[`pGR,G*@2QV!PhF"?O*RN+Boa<oe[O7Bqa^o-"eLV09l>MFt[4PS7U'l]Z.(qpG3d10peZ&#m$-Jpr]ZdI;dP`>Z6%ZL7^<lM+phf?LjHG7RslHoNnUu'nD[>/%?E&(AW8j,"XQfo<M^NaR`q(?^f",Ub<5Q~>
+Gau0E=c_<,&q6H[_1r#q!NXK5k)!@pO9D-p5S1t:FrtlCK9k..[8!o?*[;%Yrd1YjLt*22aTg=Na3ta']$@JhID<LCZbH<Y8"D%&pZ#QG'%[GA!.K7.q0h@G:3[=3Klf4s^YaHhQ,)uNM*2@]]Y:9RAanB!GNUl+0)VP@9Buh97\6qM3<+Cma\1fHcMA>"n]rKZldUp*pB.mOs5,##M5]K.B>p=o?W;AocukW*eZ-jT<peC;<(^fee(kGc#JKrRE8'Pc369G*OJh["n79.tgQ._aW"u3IcA=%3[aN-ods7`:R*k6m9XH@WD\9*B$Rq5P(/1"D?p/gE^"1ge-=Lh/G(['193<jcgHn-*A8LRl6[[!!n4=T)2NN'&X,H%'KrS[gE'*Z>_T0%+=T0GLC?3FHPR.sp&&0424Z@[sVc$[S(*$l=@5K6I0C*Q"gdQcRQ4O.#",1h1$O13*+orsXFn%V"o_qfjnuuM.:>JQ8Z>`eW4FS96W.0.G5?FtMpUW>OR%JRLR"sAEWgp>3,'TbVX!Ol8:1ucLGG?G=O,6)u*V9hSoFA]@;:N;hi'RoFo`D<cM6d.EjN$Q1D?kGM4M5T\hct2YSUSA2bG$%@=V-e4_Tke3=um1#jK+X3B,!+t4s5K84AS'".ktf'6`r5d-0,<X"M#hBF'5mSkeUsZZK*m_aDocSdaHdGaTWs5ENcft%#\N.@[(?(augeQYO\jCLRB7tgSu$gQ*RI4F<;lNHOKKQRl"r@`S?>Kg/3fMHE6?GR;[^A;Is!@b+ksJgZk^l&sS[&[Q!/C.1T!am(!Z[DIdPW.n%E%%XW`Dd?mNB9D8QdP?Ja@4.4UQ`J[pC]C<2*.I&H_fb9C:"!E6;UoqlT<PF6((#"oP3e\8mQD8,CG_mIU-gNf5[Q9c.=2k&5+Y?-r#`R7`g,]_a-.XXaeK0X*$f?"'AZe\c0X*4BWB@DH>uDljqaj+JcK3o52lgFik(cK';-*,(o8M4
 ?Kfc:q;kN>h1PYl/JJjgej[kIc2U5Wc$FJ)/CtuV/?+qBaLe;V'M:8j>klkl'.)XsTS.N7d@l0qHd</akGVK06T1=+):pCS3JAps8-'BS'8<4:(=Y2$c$$)V-L\+grgP]^H'$DQ^!d7]^"#O/gR?#.p!4NqjJc'd)$<fE>2QP$;af5bo?a2GG#,q=Zc\'<4Kd&+i'54KIHMqNiXUA!r?9`o1BC!gWQ:r@s`3]9*ok&oHo]^olR%c9C@-S331ECtSV_ZIdcq6*`l#!C.*R=oNdB6K-D=MGURFV['G&:TEE9"EDdS_\<Z5@=-Pe1s;,G6a(8h+bLh6:"bB&4C_j6C5@2"GZP7IU6fH5>BCLlV+\J\N'?3YoPCg%A`\MJO=oF@`tu?/0gUO$1>\kq+pu`E,_EU>e@JJu+:ahO"BTOWF.B?'KB\M3l;HW,aDGA.G5Ddb0<%AQ84#oceN:'pgo08j!aQE>F'@%-;SgFJ1:+5>r:@.s#[=eXV\iMMZX@$2!"oLW!QW>,et4kCiVdac^9;q#)K$OqI#hK1VcV=L1;N2KOOOJ452V*bL,b=Wp#[W]5d@Xf8:f]r!8`JlCkh3=fq`0+DD.^FjaVF^aMZ5==To%pZu3V;_'J"J-Fa)kD1;3d2PVekGM%[A2*BBs`NO60a!8rQmPcn(RlK,;,Q-KJu%-,l96@o7P>3j@ON9rEI`r^+^,J8:hY%k;s:S/*t8i#=/e;5itZkgV%&$AQB9\E2XaU.Z/>N8@R86T=brZe0s0a$CnG"Pg2,YgdNgFr:E,-gn$_6UYjM=?<pa%*hLef=B[D3:_R'CbJaSl^g2D.?L'p23aMl_^hcRb.A?foP/?!f-A_*+78W!&RL%9+R88YYbcVCp4AnCYY<sQEUs50V^<;odSj6S;E6RgD+6CTI9&R-<.ahN"CT4=TG[Rl!Kdt70YeYcN[B#YtGK(Qm:716MCoj0VhE!3h7R-+]=I9u*QeNf-EY:m.PW(
 J%VO8=L=n0cbign*9kQ^2r?&3h7Z^VhmJ!AAJK);uGc#;fZ%.^t,r.PCVJ&n]+h+S;VZT*p.+;8'$rh/Qc>d\OQ=U]?)?cu_aJ#>'NS'bh$kdQn=/t#u:"8fm2.O5_Fdcs&g;S<ZC:YmiraI1,?atW*PAK)]Bri[1Zp#K#42G:k]EkLd-[XL"@C")+-$h<B4X?+g9l"'UWbj(]V1JLcrkKKi`)2M[4ZhV8VlG\`HkJS=g<2Wtr7bj8@"IdY/R?cIbU+_1UNe9!JdWZqqO(n)k_]5%2/j_UPN2J4E?iN\C>3AB-0JoHK`pn,!]WRsU+*Irr`HpgTb7.&,GKJqZk*t/[%Gt5^aT66W'R!rum5j1sD)=Xa:q*KB,\!J1>GV\2/asdJkUE9TW9'pD4me_d$dEle0-6d&$n&AM)GXQSh]BMT+h#eo^"uc0md$_CaZ[-skm)kt=kd5iLZ\";SK[fek@Si)/b_21^+Ci~>
 endstream
 endobj
 162 0 obj
@@ -2650,53 +2650,53 @@
 xref
 0 296
 0000000000 65535 f 
-0000106200 00000 n 
-0000106465 00000 n 
-0000106558 00000 n 
+0000106207 00000 n 
+0000106472 00000 n 
+0000106565 00000 n 
 0000000015 00000 n 
 0000000071 00000 n 
 0000001295 00000 n 
 0000001415 00000 n 
 0000001573 00000 n 
-0000106710 00000 n 
+0000106717 00000 n 
 0000001708 00000 n 
-0000106773 00000 n 
+0000106780 00000 n 
 0000001845 00000 n 
-0000106839 00000 n 
+0000106846 00000 n 
 0000001982 00000 n 
-0000106905 00000 n 
+0000106912 00000 n 
 0000002119 00000 n 
-0000106969 00000 n 
+0000106976 00000 n 
 0000002255 00000 n 
-0000107035 00000 n 
+0000107042 00000 n 
 0000002392 00000 n 
-0000107101 00000 n 
+0000107108 00000 n 
 0000002529 00000 n 
-0000107165 00000 n 
+0000107172 00000 n 
 0000002665 00000 n 
-0000107229 00000 n 
+0000107236 00000 n 
 0000002802 00000 n 
-0000107293 00000 n 
+0000107300 00000 n 
 0000002939 00000 n 
-0000107357 00000 n 
+0000107364 00000 n 
 0000003074 00000 n 
-0000107424 00000 n 
+0000107431 00000 n 
 0000003211 00000 n 
-0000107489 00000 n 
+0000107496 00000 n 
 0000003348 00000 n 
-0000107554 00000 n 
+0000107561 00000 n 
 0000003483 00000 n 
-0000107621 00000 n 
+0000107628 00000 n 
 0000003620 00000 n 
-0000107688 00000 n 
+0000107695 00000 n 
 0000003757 00000 n 
-0000107755 00000 n 
+0000107762 00000 n 
 0000003893 00000 n 
-0000107820 00000 n 
+0000107827 00000 n 
 0000004030 00000 n 
-0000107887 00000 n 
+0000107894 00000 n 
 0000004167 00000 n 
-0000107952 00000 n 
+0000107959 00000 n 
 0000004304 00000 n 
 0000006935 00000 n 
 0000007058 00000 n 
@@ -2786,7 +2786,7 @@
 0000050786 00000 n 
 0000050912 00000 n 
 0000050981 00000 n 
-0000108017 00000 n 
+0000108024 00000 n 
 0000051117 00000 n 
 0000051408 00000 n 
 0000051694 00000 n 
@@ -2811,140 +2811,140 @@
 0000060215 00000 n 
 0000060412 00000 n 
 0000060610 00000 n 
-0000063159 00000 n 
-0000063285 00000 n 
-0000063322 00000 n 
-0000063533 00000 n 
-0000063731 00000 n 
-0000066482 00000 n 
-0000066608 00000 n 
-0000066677 00000 n 
-0000066901 00000 n 
-0000067106 00000 n 
-0000067363 00000 n 
-0000067553 00000 n 
-0000067758 00000 n 
-0000068005 00000 n 
-0000070617 00000 n 
-0000070743 00000 n 
-0000070876 00000 n 
-0000071077 00000 n 
-0000071288 00000 n 
-0000071496 00000 n 
-0000071671 00000 n 
-0000071866 00000 n 
-0000072040 00000 n 
-0000072216 00000 n 
-0000072439 00000 n 
-0000072680 00000 n 
-0000072876 00000 n 
-0000073089 00000 n 
-0000073306 00000 n 
-0000073525 00000 n 
-0000073797 00000 n 
-0000076526 00000 n 
-0000076652 00000 n 
-0000076729 00000 n 
-0000076990 00000 n 
-0000077266 00000 n 
-0000077475 00000 n 
-0000077796 00000 n 
-0000078114 00000 n 
-0000078300 00000 n 
-0000078498 00000 n 
-0000080763 00000 n 
-0000080873 00000 n 
-0000083234 00000 n 
-0000083344 00000 n 
-0000085676 00000 n 
-0000085786 00000 n 
-0000088161 00000 n 
-0000088271 00000 n 
-0000090486 00000 n 
-0000090596 00000 n 
-0000092693 00000 n 
-0000092803 00000 n 
-0000094015 00000 n 
-0000094125 00000 n 
-0000096001 00000 n 
-0000108077 00000 n 
-0000096111 00000 n 
-0000096247 00000 n 
-0000096440 00000 n 
-0000096598 00000 n 
-0000096814 00000 n 
-0000097098 00000 n 
-0000097268 00000 n 
-0000097418 00000 n 
-0000097594 00000 n 
-0000097910 00000 n 
-0000108131 00000 n 
-0000098100 00000 n 
-0000108198 00000 n 
-0000098294 00000 n 
-0000108263 00000 n 
-0000098486 00000 n 
-0000108330 00000 n 
-0000098701 00000 n 
-0000108397 00000 n 
-0000098869 00000 n 
-0000108464 00000 n 
-0000099076 00000 n 
-0000108529 00000 n 
-0000099280 00000 n 
-0000108595 00000 n 
-0000099457 00000 n 
-0000108662 00000 n 
-0000099697 00000 n 
-0000108729 00000 n 
-0000099894 00000 n 
-0000108795 00000 n 
-0000100091 00000 n 
-0000108863 00000 n 
-0000100270 00000 n 
-0000100476 00000 n 
-0000100697 00000 n 
-0000100981 00000 n 
-0000108931 00000 n 
-0000101314 00000 n 
-0000101480 00000 n 
-0000108997 00000 n 
-0000101695 00000 n 
-0000109063 00000 n 
-0000101871 00000 n 
-0000102059 00000 n 
-0000109131 00000 n 
-0000102280 00000 n 
-0000109197 00000 n 
-0000102525 00000 n 
-0000102713 00000 n 
-0000109265 00000 n 
-0000102984 00000 n 
-0000109333 00000 n 
-0000103148 00000 n 
-0000109399 00000 n 
-0000103375 00000 n 
-0000109467 00000 n 
-0000103530 00000 n 
-0000109533 00000 n 
-0000103751 00000 n 
-0000109601 00000 n 
-0000103942 00000 n 
-0000109669 00000 n 
-0000104195 00000 n 
-0000109737 00000 n 
-0000104440 00000 n 
-0000104631 00000 n 
-0000104900 00000 n 
-0000105070 00000 n 
-0000105255 00000 n 
-0000105420 00000 n 
-0000105534 00000 n 
-0000105645 00000 n 
-0000105757 00000 n 
-0000105866 00000 n 
-0000105973 00000 n 
-0000106090 00000 n 
+0000063166 00000 n 
+0000063292 00000 n 
+0000063329 00000 n 
+0000063540 00000 n 
+0000063738 00000 n 
+0000066489 00000 n 
+0000066615 00000 n 
+0000066684 00000 n 
+0000066908 00000 n 
+0000067113 00000 n 
+0000067370 00000 n 
+0000067560 00000 n 
+0000067765 00000 n 
+0000068012 00000 n 
+0000070624 00000 n 
+0000070750 00000 n 
+0000070883 00000 n 
+0000071084 00000 n 
+0000071295 00000 n 
+0000071503 00000 n 
+0000071678 00000 n 
+0000071873 00000 n 
+0000072047 00000 n 
+0000072223 00000 n 
+0000072446 00000 n 
+0000072687 00000 n 
+0000072883 00000 n 
+0000073096 00000 n 
+0000073313 00000 n 
+0000073532 00000 n 
+0000073804 00000 n 
+0000076533 00000 n 
+0000076659 00000 n 
+0000076736 00000 n 
+0000076997 00000 n 
+0000077273 00000 n 
+0000077482 00000 n 
+0000077803 00000 n 
+0000078121 00000 n 
+0000078307 00000 n 
+0000078505 00000 n 
+0000080770 00000 n 
+0000080880 00000 n 
+0000083241 00000 n 
+0000083351 00000 n 
+0000085683 00000 n 
+0000085793 00000 n 
+0000088168 00000 n 
+0000088278 00000 n 
+0000090493 00000 n 
+0000090603 00000 n 
+0000092700 00000 n 
+0000092810 00000 n 
+0000094022 00000 n 
+0000094132 00000 n 
+0000096008 00000 n 
+0000108084 00000 n 
+0000096118 00000 n 
+0000096254 00000 n 
+0000096447 00000 n 
+0000096605 00000 n 
+0000096821 00000 n 
+0000097105 00000 n 
+0000097275 00000 n 
+0000097425 00000 n 
+0000097601 00000 n 
+0000097917 00000 n 
+0000108138 00000 n 
+0000098107 00000 n 
+0000108205 00000 n 
+0000098301 00000 n 
+0000108270 00000 n 
+0000098493 00000 n 
+0000108337 00000 n 
+0000098708 00000 n 
+0000108404 00000 n 
+0000098876 00000 n 
+0000108471 00000 n 
+0000099083 00000 n 
+0000108536 00000 n 
+0000099287 00000 n 
+0000108602 00000 n 
+0000099464 00000 n 
+0000108669 00000 n 
+0000099704 00000 n 
+0000108736 00000 n 
+0000099901 00000 n 
+0000108802 00000 n 
+0000100098 00000 n 
+0000108870 00000 n 
+0000100277 00000 n 
+0000100483 00000 n 
+0000100704 00000 n 
+0000100988 00000 n 
+0000108938 00000 n 
+0000101321 00000 n 
+0000101487 00000 n 
+0000109004 00000 n 
+0000101702 00000 n 
+0000109070 00000 n 
+0000101878 00000 n 
+0000102066 00000 n 
+0000109138 00000 n 
+0000102287 00000 n 
+0000109204 00000 n 
+0000102532 00000 n 
+0000102720 00000 n 
+0000109272 00000 n 
+0000102991 00000 n 
+0000109340 00000 n 
+0000103155 00000 n 
+0000109406 00000 n 
+0000103382 00000 n 
+0000109474 00000 n 
+0000103537 00000 n 
+0000109540 00000 n 
+0000103758 00000 n 
+0000109608 00000 n 
+0000103949 00000 n 
+0000109676 00000 n 
+0000104202 00000 n 
+0000109744 00000 n 
+0000104447 00000 n 
+0000104638 00000 n 
+0000104907 00000 n 
+0000105077 00000 n 
+0000105262 00000 n 
+0000105427 00000 n 
+0000105541 00000 n 
+0000105652 00000 n 
+0000105764 00000 n 
+0000105873 00000 n 
+0000105980 00000 n 
+0000106097 00000 n 
 trailer
 <<
 /Size 296
@@ -2952,5 +2952,5 @@
 /Info 4 0 R
 >>
 startxref
-109805
+109812
 %%EOF

Modified: hadoop/core/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml?rev=639223&r1=639222&r2=639223&view=diff
==============================================================================
--- hadoop/core/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml (original)
+++ hadoop/core/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml Thu Mar
20 02:31:37 2008
@@ -1291,10 +1291,11 @@
           not just per task.</p> 
  
           <p>To avoid these issues the Map-Reduce framework maintains a special 
-          <code>${mapred.output.dir}/_${taskid}</code> sub-directory for each

-          task-attempt on the <code>FileSystem</code> where the output of the

-          task-attempt is stored. On successful completion of the task-attempt, 
-          the files in the <code>${mapred.output.dir}/_${taskid}</code> (only)

+          <code>${mapred.output.dir}/_temporary/_${taskid}</code> sub-directory
+          for each task-attempt on the <code>FileSystem</code> where the output
+          of the task-attempt is stored. On successful completion of the 
+          task-attempt, the files in the 
+          <code>${mapred.output.dir}/_temporary/_${taskid}</code> (only) 
           are <em>promoted</em> to <code>${mapred.output.dir}</code>.
Of course, 
           the framework discards the sub-directory of unsuccessful task-attempts. 
           This process is completely transparent to the application.</p>

Modified: hadoop/core/trunk/src/java/org/apache/hadoop/mapred/JobConf.java
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/src/java/org/apache/hadoop/mapred/JobConf.java?rev=639223&r1=639222&r2=639223&view=diff
==============================================================================
--- hadoop/core/trunk/src/java/org/apache/hadoop/mapred/JobConf.java (original)
+++ hadoop/core/trunk/src/java/org/apache/hadoop/mapred/JobConf.java Thu Mar 20 02:31:37 2008
@@ -367,10 +367,11 @@
    * <tt>task_200709221812_0001_m_000000_0</tt>), not just per TIP.</p>

    * 
    * <p>To get around this the Map-Reduce framework helps the application-writer 
-   * out by maintaining a special <tt>${mapred.output.dir}/_${taskid}</tt> 
+   * out by maintaining a special 
+   * <tt>${mapred.output.dir}/_temporary/_${taskid}</tt> 
    * sub-directory for each task-attempt on HDFS where the output of the 
    * task-attempt goes. On successful completion of the task-attempt the files 
-   * in the <tt>${mapred.output.dir}/_${taskid}</tt> (only) 
+   * in the <tt>${mapred.output.dir}/_temporary/_${taskid}</tt> (only) 
    * are <i>promoted</i> to <tt>${mapred.output.dir}</tt>. Of course,
the 
    * framework discards the sub-directory of unsuccessful task-attempts. This 
    * is completely transparent to the application.</p>
@@ -383,7 +384,7 @@
    * 
    * <p><i>Note</i>: the value of <tt>${mapred.output.dir}</tt>
during execution 
    * of a particular task-attempt is actually 
-   * <tt>${mapred.output.dir}/_{$taskid}</tt>, not the value set by 
+   * <tt>${mapred.output.dir}/_temporary/_{$taskid}</tt>, not the value set by

    * {@link #setOutputPath(Path)}. So, just create any side-files in the path 
    * returned by {@link #getOutputPath()} from map/reduce task to take 
    * advantage of this feature.</p>



Mime
View raw message