pig-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ga...@apache.org
Subject svn commit: r775340 - in /hadoop/pig/trunk: CHANGES.txt src/org/apache/pig/backend/executionengine/PigSlicer.java
Date Fri, 15 May 2009 21:08:28 GMT
Author: gates
Date: Fri May 15 21:08:27 2009
New Revision: 775340

URL: http://svn.apache.org/viewvc?rev=775340&view=rev
Log:
PIG-619: Create one InputSplit even when the input file is zero length so that hadoop runs
maps and creates output for the next job .

Modified:
    hadoop/pig/trunk/CHANGES.txt
    hadoop/pig/trunk/src/org/apache/pig/backend/executionengine/PigSlicer.java

Modified: hadoop/pig/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/pig/trunk/CHANGES.txt?rev=775340&r1=775339&r2=775340&view=diff
==============================================================================
--- hadoop/pig/trunk/CHANGES.txt (original)
+++ hadoop/pig/trunk/CHANGES.txt Fri May 15 21:08:27 2009
@@ -24,6 +24,10 @@
 
 IMPROVEMENTS
 
+PIG-619: Create one InputSplit even when the input file is zero length
+         so that hadoop runs maps and creates output for the next
+		 job (gates).
+
 PIG-693: Proposed improvements to pig's optimizer (sms)
 
 PIG-700: To automate the pig patch test process (gkesavan via sms)

Modified: hadoop/pig/trunk/src/org/apache/pig/backend/executionengine/PigSlicer.java
URL: http://svn.apache.org/viewvc/hadoop/pig/trunk/src/org/apache/pig/backend/executionengine/PigSlicer.java?rev=775340&r1=775339&r2=775340&view=diff
==============================================================================
--- hadoop/pig/trunk/src/org/apache/pig/backend/executionengine/PigSlicer.java (original)
+++ hadoop/pig/trunk/src/org/apache/pig/backend/executionengine/PigSlicer.java Fri May 15
21:08:27 2009
@@ -100,6 +100,12 @@
                 // Anything that ends with a ".gz" we must process as a complete
                 // file
                 slices.add(new PigSlice(name, funcSpec, 0, size));
+            } else if (size == 0) {
+                // add one empty slice.  This is a total hack to deal with the
+				// case where hadoop isn't starting maps for empty arrays of
+				// InputSplits.  See PIG-619.  This should be removed
+				// once we determine why this is.
+                slices.add(new PigSlice(name, funcSpec, 0, bs));
             } else {
                 while (pos < size) {
                     if (pos + bs > size) {



Mime
View raw message