hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11102) ReaderImpl: getColumnIndicesFromNames does not work for ACID tables
Date Tue, 30 Jun 2015 01:13:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606779#comment-14606779
] 

Sergey Shelukhin commented on HIVE-11102:
-----------------------------------------

The issue is actually that the column is not found.
Adding this:
{noformat}
      if (fieldNames.contains(colName)) {
        fieldIdx = fieldNames.indexOf(colName);
+     } else {
+      String s = "Cannot find field for: " + colName + " in ";
+       for (String fn : fieldNames) {
+        s += fn + ", ";
+      }
+       LOG.error(s);
+       continue;
      }
{noformat}

To one test that gets this on llap branch after merge produces

{noformat}
2015-06-29 17:45:56,629 ERROR [ORC_GET_SPLITS #2] orc.ReaderImpl: Cannot find field for: ctinyint
in _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, 
{noformat}

> ReaderImpl: getColumnIndicesFromNames does not work for ACID tables
> -------------------------------------------------------------------
>
>                 Key: HIVE-11102
>                 URL: https://issues.apache.org/jira/browse/HIVE-11102
>             Project: Hive
>          Issue Type: Bug
>          Components: File Formats
>    Affects Versions: 1.3.0, 1.2.1, 2.0.0
>            Reporter: Gopal V
>            Assignee: Sergey Shelukhin
>
> ORC reader impl does not estimate the size of ACID data files correctly.
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 0
> 	at java.util.Collections$EmptyList.get(Collections.java:3212)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcProto$Type.getSubtypes(OrcProto.java:12240)
> 	at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.getColumnIndicesFromNames(ReaderImpl.java:651)
> 	at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.getRawDataSizeOfColumns(ReaderImpl.java:634)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.populateAndCacheStripeDetails(OrcInputFormat.java:938)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:847)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:713)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:744)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message