hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6459) Change the precison/scale for intermediate sum result in the avg() udf
Date Wed, 19 Feb 2014 18:30:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13905808#comment-13905808
] 

Hive QA commented on HIVE-6459:
-------------------------------



{color:red}Overall{color}: -1 no tests executed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12629653/HIVE-6459.patch

Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1415/testReport
Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1415/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and
output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1415/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'shims/aggregator/pom.xml'
Reverted 'packaging/src/main/assembly/bin.xml'
Reverted 'conf/hive-default.xml.template'
Reverted 'bin/hive'
Reverted 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java'
Reverted 'ql/pom.xml'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/0.20S/target
shims/0.23/target shims/aggregator/target shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/hcatalog-unit/target
itests/test-serde/target itests/qtest/target itests/hive-unit/target itests/custom-serde/target
itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target hcatalog/server-extensions/target
hcatalog/core/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target
hwi/target common/target common/src/gen service/target contrib/target serde/target beeline/target
odbc/target cli/target ql/dependency-reduced-pom.xml ql/target ql/src/java/org/apache/hadoop/hive/ql/exec/HiveAuxClasspathBuilder.java
ql/src/java/org/apache/hadoop/hive/ql/exec/mr/JarCache.java
+ svn update
U    common/src/java/org/apache/hadoop/hive/common/type/UnsignedInt128.java
U    common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java
A    common/src/java/org/apache/hive/common/util/Decimal128FastBuffer.java
A    serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveDecimalWritable.java
U    serde/src/java/org/apache/hadoop/hive/serde2/io/HiveDecimalWritable.java
U    ql/src/gen/vectorization/UDAFTemplates/VectorUDAFSum.txt
A    ql/src/gen/vectorization/UDAFTemplates/VectorUDAFVarDecimal.txt
A    ql/src/gen/vectorization/UDAFTemplates/VectorUDAFMinMaxDecimal.txt
U    ql/src/gen/vectorization/UDAFTemplates/VectorUDAFVar.txt
U    ql/src/gen/vectorization/UDAFTemplates/VectorUDAFMinMax.txt
U    ql/src/gen/vectorization/UDAFTemplates/VectorUDAFAvg.txt
U    ql/src/gen/vectorization/UDAFTemplates/VectorUDAFMinMaxString.txt
A    ql/src/test/queries/clientpositive/vector_decimal_mapjoin.q
A    ql/src/test/queries/clientpositive/vector_decimal_aggregate.q
A    ql/src/test/results/clientpositive/vector_decimal_mapjoin.q.out
A    ql/src/test/results/clientpositive/vector_decimal_aggregate.q.out
U    ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java
U    ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromObjectIterables.java
U    ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java
U    ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorColumnAssignFactory.java
U    ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorExpressionDescriptor.java
U    ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorHashKeyWrapperBatch.java
U    ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatchCtx.java
U    ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapJoinOperator.java
U    ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java
U    ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorHashKeyWrapper.java
A    ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFSumDecimal.java
A    ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFAvgDecimal.java
U    ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java

Fetching external item into 'hcatalog/src/test/e2e/harness'
Updated external to revision 1569868.

Updated to revision 1569868.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12629653

> Change the precison/scale for intermediate sum result in the avg() udf 
> -----------------------------------------------------------------------
>
>                 Key: HIVE-6459
>                 URL: https://issues.apache.org/jira/browse/HIVE-6459
>             Project: Hive
>          Issue Type: Improvement
>          Components: UDF
>    Affects Versions: 0.13.0
>            Reporter: Xuefu Zhang
>            Assignee: Xuefu Zhang
>         Attachments: HIVE-6459.patch
>
>
> The avg() udf, when applied to a decimal column, selects the precision/scale of the intermediate
sum field as (p+4, s+4), which is the same for the precision/scale of the avg() result. However,
the additional scale increase is unnecessary, and the problem of data overflow may occur.
The requested change is that for the intermediate sum result,  the precsion/scale is set to
(p+10, s), which is consistent to sum() udf. The avg() result still keeps its precision/scale.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message