hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6847) Improve / fix bugs in Hive scratch dir setup
Date Fri, 22 Aug 2014 19:14:13 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107320#comment-14107320
] 

Hive QA commented on HIVE-6847:
-------------------------------



{color:red}Overall{color}: -1 no tests executed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12663616/HIVE-6847.3.patch

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/463/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/463/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-463/

Messages:
{noformat}
**** This message was trimmed, see log for full details ****
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-hcatalog-it-unit ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-hcatalog-it-unit ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-hcatalog-it-unit
---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/src/test/resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-hcatalog-it-unit ---
[INFO] Executing tasks

main:
    [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/tmp
    [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/warehouse
    [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/tmp/conf
     [copy] Copying 7 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/tmp/conf
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-hcatalog-it-unit
---
[INFO] Compiling 6 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/test-classes
[WARNING] /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/hbase/ManyMiniCluster.java:
Some input files use or override a deprecated API.
[WARNING] /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/hbase/ManyMiniCluster.java:
Recompile with -Xlint:deprecation for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-hcatalog-it-unit ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-hcatalog-it-unit ---
[INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-hcatalog-it-unit
---
[INFO] 
[INFO] --- maven-jar-plugin:2.2:test-jar (default) @ hive-hcatalog-it-unit ---
[INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar
[INFO] 
[INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-hcatalog-it-unit ---
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar
to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/pom.xml
to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.pom
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar
to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Hive Integration - Testing Utilities 0.14.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-util ---
[INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/util (includes = [datanucleus.log,
derby.log], excludes = [])
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-util ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-it-util ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/main/resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-util ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-util ---
[INFO] Compiling 49 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/classes
[WARNING] /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java:
Some input files use or override a deprecated API.
[WARNING] /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java:
Recompile with -Xlint:deprecation for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-it-util
---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/test/resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-util ---
[INFO] Executing tasks

main:
    [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp
    [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/warehouse
    [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf
     [copy] Copying 7 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-it-util ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-it-util ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-it-util ---
[INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/hive-it-util-0.14.0-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-it-util ---
[INFO] 
[INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-it-util ---
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/hive-it-util-0.14.0-SNAPSHOT.jar
to /data/hive-ptest/working/maven/org/apache/hive/hive-it-util/0.14.0-SNAPSHOT/hive-it-util-0.14.0-SNAPSHOT.jar
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/util/pom.xml to
/data/hive-ptest/working/maven/org/apache/hive/hive-it-util/0.14.0-SNAPSHOT/hive-it-util-0.14.0-SNAPSHOT.pom
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Hive Integration - Unit Tests 0.14.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-unit ---
[INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit (includes
= [datanucleus.log, derby.log], excludes = [])
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-unit ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-it-unit ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/main/resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-unit ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-unit ---
[INFO] Compiling 2 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-it-unit
---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-unit ---
[INFO] Executing tasks

main:
    [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp
    [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/warehouse
    [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/conf
     [copy] Copying 7 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/conf
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (setup-metastore-scripts) @ hive-it-unit ---
[INFO] Executing tasks

main:
    [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/scripts/metastore
     [copy] Copying 171 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/scripts/metastore
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-it-unit ---
[INFO] Compiling 70 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/test-classes
[INFO] -------------------------------------------------------------
[WARNING] COMPILATION WARNING : 
[INFO] -------------------------------------------------------------
[WARNING] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:
Some input files use or override a deprecated API.
[WARNING] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:
Recompile with -Xlint:deprecation for details.
[WARNING] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:
Some input files use unchecked or unsafe operations.
[WARNING] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:
Recompile with -Xlint:unchecked for details.
[INFO] 4 warnings 
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestUtilitiesDfs.java:[56,14]
cannot find symbol
  symbol:   method createDirsWithPermission(org.apache.hadoop.hive.conf.HiveConf,org.apache.hadoop.fs.Path,org.apache.hadoop.fs.permission.FsPermission,boolean)
  location: class org.apache.hadoop.hive.ql.exec.Utilities
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Hive Integration - Parent ......................... SUCCESS [7.366s]
[INFO] Hive Integration - Custom Serde ................... SUCCESS [11.249s]
[INFO] Hive Integration - HCatalog Unit Tests ............ SUCCESS [17.538s]
[INFO] Hive Integration - Testing Utilities .............. SUCCESS [13.056s]
[INFO] Hive Integration - Unit Tests ..................... FAILURE [13.538s]
[INFO] Hive Integration - Test Serde ..................... SKIPPED
[INFO] Hive Integration - QFile Tests .................... SKIPPED
[INFO] Hive Integration - Unit Tests - Hadoop 2 .......... SKIPPED
[INFO] Hive Integration - Unit Tests with miniKdc ........ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:07.101s
[INFO] Finished at: Fri Aug 22 15:12:30 EDT 2014
[INFO] Final Memory: 60M/160M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile
(default-testCompile) on project hive-it-unit: Compilation failure
[ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestUtilitiesDfs.java:[56,14]
cannot find symbol
[ERROR] symbol:   method createDirsWithPermission(org.apache.hadoop.hive.conf.HiveConf,org.apache.hadoop.fs.Path,org.apache.hadoop.fs.permission.FsPermission,boolean)
[ERROR] location: class org.apache.hadoop.hive.ql.exec.Utilities
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following
articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hive-it-unit
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12663616

> Improve / fix bugs in Hive scratch dir setup
> --------------------------------------------
>
>                 Key: HIVE-6847
>                 URL: https://issues.apache.org/jira/browse/HIVE-6847
>             Project: Hive
>          Issue Type: Bug
>          Components: CLI, HiveServer2
>    Affects Versions: 0.14.0
>            Reporter: Vikram Dixit K
>            Assignee: Vaibhav Gumashta
>             Fix For: 0.14.0
>
>         Attachments: HIVE-6847.1.patch, HIVE-6847.2.patch, HIVE-6847.3.patch
>
>
> Currently, the hive server creates scratch directory and changes permission to 777 however,
this is not great with respect to security. We need to create user specific scratch directories
instead. Also refer to HIVE-6782 1st iteration of the patch for approach.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message