hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raymond Lau (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-7787) Reading Parquet file with enum in Thrift Encoding throws NoSuchFieldError
Date Tue, 19 Aug 2014 22:17:18 GMT

     [ https://issues.apache.org/jira/browse/HIVE-7787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Raymond Lau updated HIVE-7787:
------------------------------

    Description: 
When reading Parquet file, where the original Thrift schema contains a struct with an enum,
this causes the following error (full stack trace blow): 
{code}
 java.lang.NoSuchFieldError: DECIMAL.
{code} 

Example Thrift Schema:
{code}
enum MyEnumType {
    EnumOne,
    EnumTwo,
    EnumThree
}

struct MyStruct {
    1: optional MyEnumType myEnumType;
    2: optional string field2;
    3: optional string field3;
}

struct outerStruct {
    1: optional list<MyStruct> myStructs
}
{code}

Error Stack trace:
{code}
Java stack trace for Hive 0.12:
Caused by: java.lang.NoSuchFieldError: DECIMAL
	at org.apache.hadoop.hive.ql.io.parquet.convert.ETypeConverter.getNewConverter(ETypeConverter.java:146)
	at org.apache.hadoop.hive.ql.io.parquet.convert.HiveGroupConverter.getConverterFromDescription(HiveGroupConverter.java:31)
	at org.apache.hadoop.hive.ql.io.parquet.convert.ArrayWritableGroupConverter.<init>(ArrayWritableGroupConverter.java:45)
	at org.apache.hadoop.hive.ql.io.parquet.convert.HiveGroupConverter.getConverterFromDescription(HiveGroupConverter.java:34)
	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:64)
	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:47)
	at org.apache.hadoop.hive.ql.io.parquet.convert.HiveGroupConverter.getConverterFromDescription(HiveGroupConverter.java:36)
	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:64)
	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:40)
	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableRecordConverter.<init>(DataWritableRecordConverter.java:32)
	at org.apache.hadoop.hive.ql.io.parquet.read.DataWritableReadSupport.prepareForRead(DataWritableReadSupport.java:128)
	at parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:142)
	at parquet.hadoop.ParquetRecordReader.initializeInternalReader(ParquetRecordReader.java:118)
	at parquet.hadoop.ParquetRecordReader.initialize(ParquetRecordReader.java:107)
	at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:92)
	at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:66)
	at org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:51)
	at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:65)
	... 16 more
{code}

  was:
When reading Parquet file, where the original Thrift schema contains an enum, this causes
the following error (full stack trace blow): 
{code}
 java.lang.NoSuchFieldError: DECIMAL.
{code} 

Example Thrift Schema:
{code}
enum MyEnumType {
    EnumOne,
    EnumTwo,
    EnumThree
}

struct MyStruct {
    1: optional MyEnumType myEnumType;
    2: optional string field2;
    3: optional string field3;
}

struct outerStruct {
    1: optional list<MyStruct> myStructs
}
{code}

Error Stack trace:
{code}
Java stack trace for Hive 0.12:
Caused by: java.lang.NoSuchFieldError: DECIMAL
	at org.apache.hadoop.hive.ql.io.parquet.convert.ETypeConverter.getNewConverter(ETypeConverter.java:146)
	at org.apache.hadoop.hive.ql.io.parquet.convert.HiveGroupConverter.getConverterFromDescription(HiveGroupConverter.java:31)
	at org.apache.hadoop.hive.ql.io.parquet.convert.ArrayWritableGroupConverter.<init>(ArrayWritableGroupConverter.java:45)
	at org.apache.hadoop.hive.ql.io.parquet.convert.HiveGroupConverter.getConverterFromDescription(HiveGroupConverter.java:34)
	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:64)
	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:47)
	at org.apache.hadoop.hive.ql.io.parquet.convert.HiveGroupConverter.getConverterFromDescription(HiveGroupConverter.java:36)
	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:64)
	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:40)
	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableRecordConverter.<init>(DataWritableRecordConverter.java:32)
	at org.apache.hadoop.hive.ql.io.parquet.read.DataWritableReadSupport.prepareForRead(DataWritableReadSupport.java:128)
	at parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:142)
	at parquet.hadoop.ParquetRecordReader.initializeInternalReader(ParquetRecordReader.java:118)
	at parquet.hadoop.ParquetRecordReader.initialize(ParquetRecordReader.java:107)
	at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:92)
	at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:66)
	at org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:51)
	at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:65)
	... 16 more
{code}


> Reading Parquet file with enum in Thrift Encoding throws NoSuchFieldError
> -------------------------------------------------------------------------
>
>                 Key: HIVE-7787
>                 URL: https://issues.apache.org/jira/browse/HIVE-7787
>             Project: Hive
>          Issue Type: Bug
>          Components: Database/Schema, Thrift API
>    Affects Versions: 0.12.0, 0.12.1
>         Environment: Hive 0.12 CDH 5.1.0, Hadoop 0.23
>            Reporter: Raymond Lau
>            Priority: Minor
>
> When reading Parquet file, where the original Thrift schema contains a struct with an
enum, this causes the following error (full stack trace blow): 
> {code}
>  java.lang.NoSuchFieldError: DECIMAL.
> {code} 
> Example Thrift Schema:
> {code}
> enum MyEnumType {
>     EnumOne,
>     EnumTwo,
>     EnumThree
> }
> struct MyStruct {
>     1: optional MyEnumType myEnumType;
>     2: optional string field2;
>     3: optional string field3;
> }
> struct outerStruct {
>     1: optional list<MyStruct> myStructs
> }
> {code}
> Error Stack trace:
> {code}
> Java stack trace for Hive 0.12:
> Caused by: java.lang.NoSuchFieldError: DECIMAL
> 	at org.apache.hadoop.hive.ql.io.parquet.convert.ETypeConverter.getNewConverter(ETypeConverter.java:146)
> 	at org.apache.hadoop.hive.ql.io.parquet.convert.HiveGroupConverter.getConverterFromDescription(HiveGroupConverter.java:31)
> 	at org.apache.hadoop.hive.ql.io.parquet.convert.ArrayWritableGroupConverter.<init>(ArrayWritableGroupConverter.java:45)
> 	at org.apache.hadoop.hive.ql.io.parquet.convert.HiveGroupConverter.getConverterFromDescription(HiveGroupConverter.java:34)
> 	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:64)
> 	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:47)
> 	at org.apache.hadoop.hive.ql.io.parquet.convert.HiveGroupConverter.getConverterFromDescription(HiveGroupConverter.java:36)
> 	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:64)
> 	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableGroupConverter.<init>(DataWritableGroupConverter.java:40)
> 	at org.apache.hadoop.hive.ql.io.parquet.convert.DataWritableRecordConverter.<init>(DataWritableRecordConverter.java:32)
> 	at org.apache.hadoop.hive.ql.io.parquet.read.DataWritableReadSupport.prepareForRead(DataWritableReadSupport.java:128)
> 	at parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:142)
> 	at parquet.hadoop.ParquetRecordReader.initializeInternalReader(ParquetRecordReader.java:118)
> 	at parquet.hadoop.ParquetRecordReader.initialize(ParquetRecordReader.java:107)
> 	at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:92)
> 	at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:66)
> 	at org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:51)
> 	at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:65)
> 	... 16 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message