parquet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jul...@apache.org
Subject incubator-parquet-mr git commit: PARQUET-135: Input location is not getting set for the getStatistics in ParquetLoader when using two different loaders within a Pig script.
Date Tue, 18 Nov 2014 00:53:21 GMT
Repository: incubator-parquet-mr
Updated Branches:
  refs/heads/master 92e6d7160 -> 251a495d2


PARQUET-135: Input location is not getting set for the getStatistics in ParquetLoader when
using two different loaders within a Pig script.

Author: elif dede <edede@twitter.com>

Closes #86 from elifdd/parquetLoader_error_PARQUET-135 and squashes the following commits:

b0150ee [elif dede] fixed white space
bdb381a [elif dede] PARQUET-135: Call setInput from getStatistics in ParquetLoader to fix
ReduceEstimator errors in pig jobs


Project: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/commit/251a495d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/tree/251a495d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/diff/251a495d

Branch: refs/heads/master
Commit: 251a495d2a72de7e892ade7f64980f51f2fcc0dd
Parents: 92e6d71
Author: elif dede <edede@twitter.com>
Authored: Mon Nov 17 16:53:08 2014 -0800
Committer: julien <julien@twitter.com>
Committed: Mon Nov 17 16:53:08 2014 -0800

----------------------------------------------------------------------
 .../parquet/format/converter/TestParquetMetadataConverter.java  | 2 +-
 parquet-pig/src/main/java/parquet/pig/ParquetLoader.java        | 5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/blob/251a495d/parquet-hadoop/src/test/java/parquet/format/converter/TestParquetMetadataConverter.java
----------------------------------------------------------------------
diff --git a/parquet-hadoop/src/test/java/parquet/format/converter/TestParquetMetadataConverter.java
b/parquet-hadoop/src/test/java/parquet/format/converter/TestParquetMetadataConverter.java
index 0f7dccf..367f5d3 100644
--- a/parquet-hadoop/src/test/java/parquet/format/converter/TestParquetMetadataConverter.java
+++ b/parquet-hadoop/src/test/java/parquet/format/converter/TestParquetMetadataConverter.java
@@ -244,7 +244,7 @@ public class TestParquetMetadataConverter {
       try {
         verifyAllFilters(metadata(rgs), splitSize);
       } catch (AssertionError e) {
-        throw new AssertionError("fail verifyAllFilters(metadata(" + Arrays.toString(rgs)
+ "), " + splitSize + ")", e);
+	  throw new AssertionError("fail verifyAllFilters(metadata(" + Arrays.toString(rgs) + "),
" + splitSize + ")", e);
       }
     }
   }

http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/blob/251a495d/parquet-pig/src/main/java/parquet/pig/ParquetLoader.java
----------------------------------------------------------------------
diff --git a/parquet-pig/src/main/java/parquet/pig/ParquetLoader.java b/parquet-pig/src/main/java/parquet/pig/ParquetLoader.java
index 2e96ecf..7bf70e9 100644
--- a/parquet-pig/src/main/java/parquet/pig/ParquetLoader.java
+++ b/parquet-pig/src/main/java/parquet/pig/ParquetLoader.java
@@ -287,8 +287,9 @@ public class ParquetLoader extends LoadFunc implements LoadMetadata, LoadPushDow
   public ResourceStatistics getStatistics(String location, Job job)
       throws IOException {
     if (DEBUG) LOG.debug("LoadMetadata.getStatistics(" + location + ", " + job + ")");
-    // We do not need to call setInput
-    // as setLocation is guaranteed to be called before this
+    /* We need to call setInput since setLocation is not
+       guaranteed to be called before this */
+    setInput(location, job);
     long length = 0;
     try {
       for (InputSplit split : getParquetInputFormat().getSplits(job)) {


Mime
View raw message