hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thiruvel Thirumoolan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-8417) round(decimal, negative) errors out/wrong results with reduce side vectorization
Date Thu, 09 Oct 2014 09:31:34 GMT
Thiruvel Thirumoolan created HIVE-8417:
------------------------------------------

             Summary: round(decimal, negative) errors out/wrong results with reduce side vectorization
                 Key: HIVE-8417
                 URL: https://issues.apache.org/jira/browse/HIVE-8417
             Project: Hive
          Issue Type: Bug
          Components: Vectorization
    Affects Versions: 0.14.0
            Reporter: Thiruvel Thirumoolan
            Assignee: Jitendra Nath Pandey
            Priority: Critical


With reduce-side vectorization enabled, round UDF taking a decimal value and a negative argument
fails. It passes when there is no reducer or when vectorization is turned off.

Simulated with:
create table decimal_tbl (dec decimal(10,0));

Data: just one record, "101"

Query: select dec, round(dec, -1) from decimal_tbl order by dec;

This query fails with text and rcfile with IndexOutOfBoundsException in Decimal128.toFormalString(),
but returns "101 101" with orc. When order by is removed, it returns "101 100" with orc and
rc. When "order by dec" is replaced with "order by round(dec, -1) it fails with the same exception
with orc too.

Following is the exception thrown:

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing
vector batch (tag=0) [Error getting row data with exception java.lang.IndexOutOfBoundsException:
start 0, end 3, s.length() 2
	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:476)
	at java.lang.StringBuilder.append(StringBuilder.java:191)
	at org.apache.hadoop.hive.common.type.Decimal128.toFormalString(Decimal128.java:1858)
	at org.apache.hadoop.hive.common.type.Decimal128.toBigDecimal(Decimal128.java:1733)
	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$1.writeValue(VectorExpressionWriterFactory.java:469)
	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterDecimal.writeValue(VectorExpressionWriterFactory.java:310)
	at org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch.toString(VectorizedRowBatch.java:159)
	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectors(ReduceRecordSource.java:371)
	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:250)
	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:168)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:142)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1637)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:722)
 ]
	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectors(ReduceRecordSource.java:376)
	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:250)
	... 16 more
Caused by: java.lang.IndexOutOfBoundsException: start 0, end 3, s.length() 2
	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:476)
	at java.lang.StringBuilder.append(StringBuilder.java:191)
	at org.apache.hadoop.hive.common.type.Decimal128.toFormalString(Decimal128.java:1858)
	at org.apache.hadoop.hive.common.type.Decimal128.toBigDecimal(Decimal128.java:1733)
	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$1.writeValue(VectorExpressionWriterFactory.java:469)
	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterDecimal.writeValue(VectorExpressionWriterFactory.java:310)
	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterSetter.writeValue(VectorExpressionWriterFactory.java:1153)
	at org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.getRowObject(VectorFileSinkOperator.java:76)
	at org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.processOp(VectorFileSinkOperator.java:63)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:799)
	at org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectors(ReduceRecordSource.java:360)
	... 17 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message