parquet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From b...@apache.org
Subject parquet-mr git commit: PARQUET-152: Add validation on Encoding.DELTA_BYTE_ARRAY to allow FIX…
Date Fri, 03 Jul 2015 17:51:43 GMT
Repository: parquet-mr
Updated Branches:
  refs/heads/master c334a1bca -> 013b445ed


PARQUET-152: Add validation on Encoding.DELTA_BYTE_ARRAY to allow FIX…

PARQUET-152: Add validation on Encoding.DELTA_BYTE_ARRAY to allow FIXED_LEN_BYTE_ARRAY types.

  * FIXED_LEN_BYTE_ARRAY types are binary values that may use DELTA_BYTE_ARRAY encoding,
    so they should be allowed to be decoded using the same DELTA_BYTE_ARRAY encoding.

@rdblue @nezihyigitbasi  Could you review this fix?

I executed a test by writing a file that fall backs to DELTA_BYTE_ARRAY encoding, then read
the file, and compare the read values with the written values, and it worked fine.

Author: Sergio Pena <sergio.pena@cloudera.com>

Closes #225 from spena/parquet-152 and squashes the following commits:

93fa03e [Sergio Pena] PARQUET-152: Add validation on Encoding.DELTA_BYTE_ARRAY to allow FIXED_LEN_BYTE_ARRAY
types.


Project: http://git-wip-us.apache.org/repos/asf/parquet-mr/repo
Commit: http://git-wip-us.apache.org/repos/asf/parquet-mr/commit/013b445e
Tree: http://git-wip-us.apache.org/repos/asf/parquet-mr/tree/013b445e
Diff: http://git-wip-us.apache.org/repos/asf/parquet-mr/diff/013b445e

Branch: refs/heads/master
Commit: 013b445ede8d9e7aad4915859d0c869b9b712f8d
Parents: c334a1b
Author: Sergio Pena <sergio.pena@cloudera.com>
Authored: Fri Jul 3 10:51:34 2015 -0700
Committer: Ryan Blue <blue@apache.org>
Committed: Fri Jul 3 10:51:34 2015 -0700

----------------------------------------------------------------------
 .../src/main/java/org/apache/parquet/column/Encoding.java       | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/parquet-mr/blob/013b445e/parquet-column/src/main/java/org/apache/parquet/column/Encoding.java
----------------------------------------------------------------------
diff --git a/parquet-column/src/main/java/org/apache/parquet/column/Encoding.java b/parquet-column/src/main/java/org/apache/parquet/column/Encoding.java
index 9770044..87bc798 100644
--- a/parquet-column/src/main/java/org/apache/parquet/column/Encoding.java
+++ b/parquet-column/src/main/java/org/apache/parquet/column/Encoding.java
@@ -19,6 +19,7 @@
 package org.apache.parquet.column;
 
 import static org.apache.parquet.column.values.bitpacking.Packer.BIG_ENDIAN;
+import static org.apache.parquet.schema.PrimitiveType.PrimitiveTypeName.FIXED_LEN_BYTE_ARRAY;
 import static org.apache.parquet.schema.PrimitiveType.PrimitiveTypeName.INT32;
 import static org.apache.parquet.schema.PrimitiveType.PrimitiveTypeName.BINARY;
 import static org.apache.parquet.schema.PrimitiveType.PrimitiveTypeName.BOOLEAN;
@@ -192,8 +193,8 @@ public enum Encoding {
     @Override
     public ValuesReader getValuesReader(ColumnDescriptor descriptor,
         ValuesType valuesType) {
-      if (descriptor.getType() != BINARY) {
-        throw new ParquetDecodingException("Encoding DELTA_BYTE_ARRAY is only supported for
type BINARY");
+      if (descriptor.getType() != BINARY && descriptor.getType() != FIXED_LEN_BYTE_ARRAY)
{
+        throw new ParquetDecodingException("Encoding DELTA_BYTE_ARRAY is only supported for
type BINARY and FIXED_LEN_BYTE_ARRAY");
       }
       return new DeltaByteArrayReader();
     }


Mime
View raw message