hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suma Shivaprasad (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7629) Problem in SMB Joins between two Parquet tables
Date Wed, 06 Aug 2014 07:23:11 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087366#comment-14087366
] 

Suma Shivaprasad commented on HIVE-7629:
----------------------------------------

The issue is due to DataWritableReadSupport trying to access column indexes which are not
present in the table. Have the fix for the same. Will be attaching the patch for it shortly


> Problem in SMB Joins between two Parquet tables
> -----------------------------------------------
>
>                 Key: HIVE-7629
>                 URL: https://issues.apache.org/jira/browse/HIVE-7629
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>    Affects Versions: 0.13.1
>            Reporter: Suma Shivaprasad
>
> The issue is clearly seen when two bucketed and sorted parquet tables with different
number of columns are involved in the join . The following exception is seen
> Caused by: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
>         at java.util.ArrayList.rangeCheck(ArrayList.java:635)
>         at java.util.ArrayList.get(ArrayList.java:411)
>         at org.apache.hadoop.hive.ql.io.parquet.read.DataWritableReadSupport.init(DataWritableReadSupport.java:101)
>         at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.getSplit(ParquetRecordReaderWrapper.java:204)
>         at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:79)
>         at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:66)
>         at org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:51)
>         at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:65)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message