hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Vary (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14444) Upgrade qtest execution framework to junit4 - migrate most of them
Date Fri, 12 Aug 2016 23:17:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419671#comment-15419671
] 

Peter Vary commented on HIVE-14444:
-----------------------------------

I am only concerned about the end results, and not really about the steps which are leading
there. So if you agree, after you commit the patch then I will incorporate the stuff I proposed
in the review. Of course I would like you both to review my patch.

Regarding of the BeeLine testcases, I think we should check the HIVE-4161. It was reported
be [~thejas] a log time ago, and it was about to creating a small subset of the test cases
which should use BeeLine for testing Hive. The funny thing is, that it was only partially
solved - with the result of removing every single BeeLine tests in the end, and we are still
waiting for someone to create some BeeLine tests package.

On the other hand, in this document we stated, that the HiveCLI is deprecated, and the users
should use BeeLine instead (https://cwiki.apache.org/confluence/display/Hive/Replacing+the+Implementation+of+Hive+CLI+Using+Beeline)

I think with that in mind, in the medium term we should consider to migrate the integration
test cases to BeeLine. The exact method and steps is up for discussion. Maybe the first thing
would be to create the working TestBeeLine testcase for several chosen queries from the existing
mix, and with that check the viability of the idea. It this is possible we might consider
running the testcases parallel. This has serious risks, but can have a huge impact on the
overall runtime of the tests.

I do not think I could like to create a patch independently of this one to make the BeeLine
testcases working. What I did do to be able to run the alter3.q testcase query on my laptop
is some trivial refactoring, and the following almost trivial changes:
{noformat}
diff --git beeline/src/java/org/apache/hive/beeline/util/QFileClient.java beeline/src/java/org/apache/hive/beeline/util/QFileClient.java
index 81f1b0e..44b9ca5 100644
--- beeline/src/java/org/apache/hive/beeline/util/QFileClient.java
+++ beeline/src/java/org/apache/hive/beeline/util/QFileClient.java
@@ -125,6 +125,7 @@ void initFilterSet() {
     .addFilter(outputDirectory.toString(), "!!{outputDirectory}!!")
     .addFilter(qFileDirectory.toString(), "!!{qFileDirectory}!!")
     .addFilter(hiveRootDirectory.toString(), "!!{hive.root}!!")
+    .addFilter("\\(queryId=[^\\)]*\\)","queryId=(!!{queryId}!!)")
     .addFilter("file:/\\w\\S+", "file:/!!ELIDED!!")
     .addFilter("pfile:/\\w\\S+", "pfile:/!!ELIDED!!")
     .addFilter("hdfs:/\\w\\S+", "hdfs:/!!ELIDED!!")
@@ -134,6 +135,7 @@ void initFilterSet() {
     .addFilter("(\\D)" + currentTimePrefix + "\\d{9}(\\D)", "$1!!UNIXTIMEMILLIS!!$2")
     .addFilter(userName, "!!{user.name}!!")
     .addFilter(operatorPattern, "\"$1_!!ELIDED!!\"")
+    .addFilter("Time taken: [0-9\\.]* seconds", "Time taken: !!ELIDED!! seconds")
     ;
   };
{noformat}

With that I think we will have a more or less working testdriver for BeeLine testcase, on
which we can improve on.

> Upgrade qtest execution framework to junit4 - migrate most of them
> ------------------------------------------------------------------
>
>                 Key: HIVE-14444
>                 URL: https://issues.apache.org/jira/browse/HIVE-14444
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Tests
>            Reporter: Zoltan Haindrich
>            Assignee: Zoltan Haindrich
>         Attachments: HIVE-14444.1.patch, HIVE-14444.2.patch, HIVE-14444.3.patch, HIVE-14444.4.patch
>
>
> this is the second step..migrating all exiting qtestgen generated tests to junit4
> it might be possible that not all will get migrated in this ticket...I will leave out
the problematic ones...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message