Return-Path: X-Original-To: apmail-hive-issues-archive@minotaur.apache.org Delivered-To: apmail-hive-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 650AB102E6 for ; Sat, 12 Sep 2015 04:25:46 +0000 (UTC) Received: (qmail 69671 invoked by uid 500); 12 Sep 2015 04:25:46 -0000 Delivered-To: apmail-hive-issues-archive@hive.apache.org Received: (qmail 69619 invoked by uid 500); 12 Sep 2015 04:25:46 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 69604 invoked by uid 99); 12 Sep 2015 04:25:45 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 12 Sep 2015 04:25:45 +0000 Date: Sat, 12 Sep 2015 04:25:45 +0000 (UTC) From: "Hive QA (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-11792) User explain in tez does not preserve ordering MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-11792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14741896#comment-14741896 ] Hive QA commented on HIVE-11792: -------------------------------- {color:red}Overall{color}: -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12755254/HIVE-11792.1.patch Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5248/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5248/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5248/ Messages: {noformat} **** This message was trimmed, see log for full details **** [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-shims --- [INFO] Installing /data/hive-ptest/working/apache-github-source-source/shims/aggregator/target/hive-shims-2.0.0-SNAPSHOT.jar to /home/hiveptest/.m2/repository/org/apache/hive/hive-shims/2.0.0-SNAPSHOT/hive-shims-2.0.0-SNAPSHOT.jar [INFO] Installing /data/hive-ptest/working/apache-github-source-source/shims/aggregator/pom.xml to /home/hiveptest/.m2/repository/org/apache/hive/hive-shims/2.0.0-SNAPSHOT/hive-shims-2.0.0-SNAPSHOT.pom [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Hive Storage API 2.0.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-storage-api --- [INFO] Deleting /data/hive-ptest/working/apache-github-source-source/storage-api/target [INFO] Deleting /data/hive-ptest/working/apache-github-source-source/storage-api (includes = [datanucleus.log, derby.log], excludes = []) [INFO] [INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @ hive-storage-api --- [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-storage-api --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-storage-api --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-github-source-source/storage-api/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-storage-api --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-storage-api --- [INFO] Compiling 13 source files to /data/hive-ptest/working/apache-github-source-source/storage-api/target/classes [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-storage-api --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-github-source-source/storage-api/src/test/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-storage-api --- [INFO] Executing tasks main: [mkdir] Created dir: /data/hive-ptest/working/apache-github-source-source/storage-api/target/tmp [mkdir] Created dir: /data/hive-ptest/working/apache-github-source-source/storage-api/target/warehouse [mkdir] Created dir: /data/hive-ptest/working/apache-github-source-source/storage-api/target/tmp/conf [copy] Copying 10 files to /data/hive-ptest/working/apache-github-source-source/storage-api/target/tmp/conf [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-storage-api --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-storage-api --- [INFO] Tests are skipped. [INFO] [INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-storage-api --- [INFO] Building jar: /data/hive-ptest/working/apache-github-source-source/storage-api/target/hive-storage-api-2.0.0-SNAPSHOT.jar [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-storage-api --- [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-storage-api --- [INFO] Installing /data/hive-ptest/working/apache-github-source-source/storage-api/target/hive-storage-api-2.0.0-SNAPSHOT.jar to /home/hiveptest/.m2/repository/org/apache/hive/hive-storage-api/2.0.0-SNAPSHOT/hive-storage-api-2.0.0-SNAPSHOT.jar [INFO] Installing /data/hive-ptest/working/apache-github-source-source/storage-api/pom.xml to /home/hiveptest/.m2/repository/org/apache/hive/hive-storage-api/2.0.0-SNAPSHOT/hive-storage-api-2.0.0-SNAPSHOT.pom [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Hive Common 2.0.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-common --- [INFO] Deleting /data/hive-ptest/working/apache-github-source-source/common/target [INFO] Deleting /data/hive-ptest/working/apache-github-source-source/common (includes = [datanucleus.log, derby.log], excludes = []) [INFO] [INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @ hive-common --- [INFO] [INFO] --- maven-antrun-plugin:1.7:run (generate-version-annotation) @ hive-common --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- build-helper-maven-plugin:1.8:add-source (add-source) @ hive-common --- [INFO] Source directory: /data/hive-ptest/working/apache-github-source-source/common/src/gen added. [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-common --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-common --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 1 resource [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-common --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-common --- [INFO] Compiling 76 source files to /data/hive-ptest/working/apache-github-source-source/common/target/classes [INFO] ------------------------------------------------------------- [WARNING] COMPILATION WARNING : [INFO] ------------------------------------------------------------- [WARNING] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/JvmPauseMonitor.java: /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/JvmPauseMonitor.java uses or overrides a deprecated API. [WARNING] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/JvmPauseMonitor.java: Recompile with -Xlint:deprecation for details. [WARNING] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/ObjectPair.java: Some input files use unchecked or unsafe operations. [WARNING] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/ObjectPair.java: Recompile with -Xlint:unchecked for details. [INFO] 4 warnings [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Vertex.java:[180,15] cannot find symbol symbol: variable printSet location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Vertex.java:[190,11] cannot find symbol symbol: variable printSet location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Op.java:[130,15] cannot find symbol symbol: variable printSet location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Op.java:[135,11] cannot find symbol symbol: variable printSet location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Stage.java:[229,15] cannot find symbol symbol: variable printSet location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Stage.java:[234,11] cannot find symbol symbol: variable printSet location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [INFO] 6 errors [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Hive .............................................. SUCCESS [13.468s] [INFO] Hive Shims Common ................................. SUCCESS [15.995s] [INFO] Hive Shims 0.20S .................................. SUCCESS [3.519s] [INFO] Hive Shims 0.23 ................................... SUCCESS [10.711s] [INFO] Hive Shims Scheduler .............................. SUCCESS [2.260s] [INFO] Hive Shims ........................................ SUCCESS [2.718s] [INFO] Hive Storage API .................................. SUCCESS [2.553s] [INFO] Hive Common ....................................... FAILURE [17.592s] [INFO] Hive Serde ........................................ SKIPPED [INFO] Hive Metastore .................................... SKIPPED [INFO] Hive Ant Utilities ................................ SKIPPED [INFO] Spark Remote Client ............................... SKIPPED [INFO] Hive Query Language ............................... SKIPPED [INFO] Hive Service ...................................... SKIPPED [INFO] Hive Accumulo Handler ............................. SKIPPED [INFO] Hive JDBC ......................................... SKIPPED [INFO] Hive Beeline ...................................... SKIPPED [INFO] Hive CLI .......................................... SKIPPED [INFO] Hive Contrib ...................................... SKIPPED [INFO] Hive HBase Handler ................................ SKIPPED [INFO] Hive HCatalog ..................................... SKIPPED [INFO] Hive HCatalog Core ................................ SKIPPED [INFO] Hive HCatalog Pig Adapter ......................... SKIPPED [INFO] Hive HCatalog Server Extensions ................... SKIPPED [INFO] Hive HCatalog Webhcat Java Client ................. SKIPPED [INFO] Hive HCatalog Webhcat ............................. SKIPPED [INFO] Hive HCatalog Streaming ........................... SKIPPED [INFO] Hive HPL/SQL ...................................... SKIPPED [INFO] Hive HWI .......................................... SKIPPED [INFO] Hive ODBC ......................................... SKIPPED [INFO] Hive Shims Aggregator ............................. SKIPPED [INFO] Hive TestUtils .................................... SKIPPED [INFO] Hive Packaging .................................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1:11.892s [INFO] Finished at: Sat Sep 12 00:25:09 EDT 2015 [INFO] Final Memory: 50M/164M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hive-common: Compilation failure: Compilation failure: [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Vertex.java:[180,15] cannot find symbol [ERROR] symbol: variable printSet [ERROR] location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Vertex.java:[190,11] cannot find symbol [ERROR] symbol: variable printSet [ERROR] location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Op.java:[130,15] cannot find symbol [ERROR] symbol: variable printSet [ERROR] location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Op.java:[135,11] cannot find symbol [ERROR] symbol: variable printSet [ERROR] location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Stage.java:[229,15] cannot find symbol [ERROR] symbol: variable printSet [ERROR] location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [ERROR] /data/hive-ptest/working/apache-github-source-source/common/src/java/org/apache/hadoop/hive/common/jsonexplain/tez/Stage.java:[234,11] cannot find symbol [ERROR] symbol: variable printSet [ERROR] location: variable parser of type org.apache.hadoop.hive.common.jsonexplain.tez.TezJsonParser [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :hive-common + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12755254 - PreCommit-HIVE-TRUNK-Build > User explain in tez does not preserve ordering > ---------------------------------------------- > > Key: HIVE-11792 > URL: https://issues.apache.org/jira/browse/HIVE-11792 > Project: Hive > Issue Type: Bug > Components: Diagnosability > Affects Versions: 2.0.0 > Reporter: Prasanth Jayachandran > Assignee: Prasanth Jayachandran > Attachments: HIVE-11792.1.patch > > > HIVE-11037 introduced user level explain. The output of operator attributes are JSONObject which does not preserve ordering. This is causing diffs in test output when run under tez vs llap like below > {code} > - partition:{"ts":"2012-01-03+14:46:31","ds":"2012-01-03"} > - table:{"serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde","name:":"default.src_orc_merge_test_part","input format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat"} > + partition:{"ds":"2012-01-03","ts":"2012-01-03+14:46:31"} > + table:{"name:":"default.src_orc_merge_test_part","input format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat","serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde"} > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)