hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth J (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7698) TestMiniTezCliDriver timing out
Date Tue, 12 Aug 2014 22:30:14 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14094805#comment-14094805
] 

Prasanth J commented on HIVE-7698:
----------------------------------

[~brocknoland] you're welcome!

> TestMiniTezCliDriver timing out
> -------------------------------
>
>                 Key: HIVE-7698
>                 URL: https://issues.apache.org/jira/browse/HIVE-7698
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Brock Noland
>
> When testing HIVE-7629 in [build 279|http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/279/consoleFull]
I noticed that the test was taking much longer than normal (2 hours instead of 1) and that
TestMiniTezCliDriver was the only test still running. Below is the process listing.
> As you can see below, it was running the following .q files: 
> {noformat}
> tez_joins_explain.q
> vector_decimal_aggregate.q
> mapjoin_mapjoin.q
> groupby2.q
> join1.q
> bucket_map_join_tez1.q
> tez_union.q
> tez_dml.q
> dynpart_sort_optimization.q
> orc_merge2.q
> load_dyn_part1.q
> disable_merge_for_bucketing.q
> filter_join_breaktask2.q
> ctas.q
> orc_merge3.q
> {noformat}
> {noformat}
> 502      15573  0.0  0.0   9236  1224 ?        Ss   16:11   0:00  |       \_ bash /home/hiveptest//54.221.124.118-hiveptest-2/scratch/hiveptest-TestMiniTezCliDriver-tez_joins_explain.q-vector_decimal_aggregate.q-mapjoin_mapjoin.q-and-12-more.sh
> 502      15628  0.0  0.0   4128   636 ?        S    16:11   0:00  |           \_ timeout
2h mvn -B -o test -Dmaven.repo.local=/home/hiveptest//54.221.124.118-hiveptest-2/maven -Phadoop-2
-Phadoop-2 -Dtest=TestMiniTezCliDriver -Dqfile=tez_joins_explain.q,vector_decimal_aggregate.q,mapjoin_mapjoin.q,groupby2.q,join1.q,bucket_map_join_tez1.q,tez_union.q,tez_dml.q,dynpart_sort_optimization.q,orc_merge2.q,load_dyn_part1.q,disable_merge_for_bucketing.q,filter_join_breaktask2.q,ctas.q,orc_merge3.q
> 502      15629  0.6  4.5 4375516 703136 ?      Sl   16:11   0:42  |               \_
/usr/java/jdk1.7.0_45-cloudera/bin/java -Xmx2g -XX:MaxPermSize=256M -classpath /usr/local/apache-maven-3.0.5/boot/plexus-classworlds-2.4.jar
-Dclassworlds.conf=/usr/local/apache-maven-3.0.5/bin/m2.conf -Dmaven.home=/usr/local/apache-maven-3.0.5
org.codehaus.plexus.classworlds.launcher.Launcher -B -o test -Dmaven.repo.local=/home/hiveptest//54.221.124.118-hiveptest-2/maven
-Phadoop-2 -Phadoop-2 -Dtest=TestMiniTezCliDriver -Dqfile=tez_joins_explain.q,vector_decimal_aggregate.q,mapjoin_mapjoin.q,groupby2.q,join1.q,bucket_map_join_tez1.q,tez_union.q,tez_dml.q,dynpart_sort_optimization.q,orc_merge2.q,load_dyn_part1.q,disable_merge_for_bucketing.q,filter_join_breaktask2.q,ctas.q,orc_merge3.q
> 502      15866  0.0  0.0 106112  1264 ?        S    16:12   0:00  |                 
 \_ /bin/sh -c cd /home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest
&& /usr/java/jdk1.7.0_45-cloudera/jre/bin/java -Xmx2048m -XX:MaxPermSize=512m -jar
/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/surefire/surefirebooter8865040196821476054.jar
/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/surefire/surefire2549852945539040510tmp
/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/surefire/surefire_08576952877364320449tmp
> 502      15867  3.3  4.2 7613168 647740 ?      Sl   16:12   3:31  |                 
     \_ /usr/java/jdk1.7.0_45-cloudera/jre/bin/java -Xmx2048m -XX:MaxPermSize=512m -jar /home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/surefire/surefirebooter8865040196821476054.jar
/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/surefire/surefire2549852945539040510tmp
/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/surefire/surefire_08576952877364320449tmp
> 502       1426  0.0  0.0 106264  1332 ?        Ss   16:14   0:00  |                 
         \_ /bin/bash -c /usr/java/jdk1.7.0_45-cloudera/bin/java -Dlog4j.configuration=tez-container-log4j.properties
-Dyarn.app.container.log.dir=/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/hive/hive-logDir-nm-1_0/application_1407874332889_0006/container_1407874332889_0006_01_000002
-Dtez.root.logger=INFO,CLA  -Xmx1024m  -Dsun.nio.ch.bugLevel='' org.apache.tez.dag.app.DAGAppMaster
--session 1>/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/hive/hive-logDir-nm-1_0/application_1407874332889_0006/container_1407874332889_0006_01_000002/stdout
2>/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/hive/hive-logDir-nm-1_0/application_1407874332889_0006/container_1407874332889_0006_01_000002/stderr

> 502       1442  0.7  1.7 5626068 272824 ?      Sl   16:14   0:47  |                 
         |   \_ /usr/java/jdk1.7.0_45-cloudera/bin/java -Dlog4j.configuration=tez-container-log4j.properties
-Dyarn.app.container.log.dir=/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/hive/hive-logDir-nm-1_0/application_1407874332889_0006/container_1407874332889_0006_01_000002
-Dtez.root.logger=INFO,CLA -Xmx1024m -Dsun.nio.ch.bugLevel= org.apache.tez.dag.app.DAGAppMaster
--session
> 502       2828  0.0  0.0 106268  1348 ?        Ss   16:15   0:00  |                 
         \_ /bin/bash -c /usr/java/jdk1.7.0_45-cloudera/bin/java -Djava.net.preferIPv4Stack=true
-Dhadoop.metrics.log.level=WARN -Xmx200m -Dlog4j.configuration=tez-container-log4j.properties
-Dyarn.app.container.log.dir=/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/hive/hive-logDir-nm-0_3/application_1407874332889_0006/container_1407874332889_0006_01_000003
-Dtez.root.logger=INFO,CLA  -Djava.io.tmpdir=/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/hive/hive-localDir-nm-0_1/usercache/hiveptest/appcache/application_1407874332889_0006/container_1407874332889_0006_01_000003/tmp
org.apache.hadoop.mapred.YarnTezDagChild 10.69.139.135 55252 container_1407874332889_0006_01_000003
application_1407874332889_0006 1 1>/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/hive/hive-logDir-nm-0_3/application_1407874332889_0006/container_1407874332889_0006_01_000003/stdout
2>/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/hive/hive-logDir-nm-0_3/application_1407874332889_0006/container_1407874332889_0006_01_000003/stderr

> 502       2842  0.6  1.5 3467232 238608 ?      Sl   16:15   0:40  |                 
             \_ /usr/java/jdk1.7.0_45-cloudera/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN
-Xmx200m -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/hive/hive-logDir-nm-0_3/application_1407874332889_0006/container_1407874332889_0006_01_000003
-Dtez.root.logger=INFO,CLA -Djava.io.tmpdir=/home/hiveptest/54.221.124.118-hiveptest-2/apache-svn-trunk-source/itests/qtest/target/hive/hive-localDir-nm-0_1/usercache/hiveptest/appcache/application_1407874332889_0006/container_1407874332889_0006_01_000003/tmp
org.apache.hadoop.mapred.YarnTezDagChild 10.69.139.135 55252 container_1407874332889_0006_01_000003
application_1407874332889_0006 1
> {noformat}
> The test result which was impacted by this is here: https://issues.apache.org/jira/browse/HIVE-7629?focusedCommentId=14094778&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14094778



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message