hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt McCline (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11981) ORC Schema Evolution Issues (Vectorized, ACID, and Non-Vectorized)
Date Wed, 11 Nov 2015 21:19:10 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15001106#comment-15001106
] 

Matt McCline commented on HIVE-11981:
-------------------------------------

transform_acid.q Hive QA hive.log:

{code}
java.lang.Exception: java.lang.RuntimeException: Hive Runtime Error while closing operators
	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) ~[hadoop-mapreduce-client-common-2.6.0.jar:?]
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) [hadoop-mapreduce-client-common-2.6.0.jar:?]
Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators
	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:204) ~[hive-exec-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61) ~[hadoop-mapreduce-client-core-2.6.0.jar:?]
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450) ~[hadoop-mapreduce-client-core-2.6.0.jar:?]
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) ~[hadoop-mapreduce-client-core-2.6.0.jar:?]
	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
~[hadoop-mapreduce-client-common-2.6.0.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[?:1.7.0_45]
	at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[?:1.7.0_45]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[?:1.7.0_45]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[?:1.7.0_45]
	at java.lang.Thread.run(Thread.java:744) ~[?:1.7.0_45]
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: [Error 20003]: An error occurred
when trying to close the Operator running your custom script.
	at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:562) ~[hive-exec-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:670) ~[hive-exec-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:670) ~[hive-exec-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:670) ~[hive-exec-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:186) ~[hive-exec-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61) ~[hadoop-mapreduce-client-core-2.6.0.jar:?]
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450) ~[hadoop-mapreduce-client-core-2.6.0.jar:?]
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) ~[hadoop-mapreduce-client-core-2.6.0.jar:?]
	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
~[hadoop-mapreduce-client-common-2.6.0.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[?:1.7.0_45]
	at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[?:1.7.0_45]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[?:1.7.0_45]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[?:1.7.0_45]
	at java.lang.Thread.run(Thread.java:744) ~[?:1.7.0_45]
{code}

It doesn't fail on my laptop.

> ORC Schema Evolution Issues (Vectorized, ACID, and Non-Vectorized)
> ------------------------------------------------------------------
>
>                 Key: HIVE-11981
>                 URL: https://issues.apache.org/jira/browse/HIVE-11981
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive, Transactions
>            Reporter: Matt McCline
>            Assignee: Matt McCline
>            Priority: Critical
>         Attachments: HIVE-11981.01.patch, HIVE-11981.02.patch, HIVE-11981.03.patch, HIVE-11981.05.patch,
HIVE-11981.06.patch, HIVE-11981.07.patch, HIVE-11981.08.patch, HIVE-11981.09.patch, HIVE-11981.091.patch,
HIVE-11981.092.patch, HIVE-11981.093.patch, HIVE-11981.094.patch, HIVE-11981.095.patch, HIVE-11981.096.patch,
HIVE-11981.097.patch, HIVE-11981.098.patch, ORC Schema Evolution Issues.docx
>
>
> High priority issues with schema evolution for the ORC file format.
> Schema evolution here is limited to adding new columns and a few cases of column type-widening
(e.g. int to bigint).
> Renaming columns, deleting column, moving columns and other schema evolution were not
pursued due to lack of importance and lack of time.  Also, it appears a much more sophisticated
metadata would be needed to support them.
> The biggest issues for users have been adding new columns for ACID table (HIVE-11421
Support Schema evolution for ACID tables) and vectorization (HIVE-10598 Vectorization borks
when column is added to table).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message