hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Vary <pv...@cloudera.com>
Subject Re: Review Request 57693: HIVE-16146 If possible find a better way to filter the TestBeeLineDriver outpu
Date Fri, 07 Apr 2017 09:38:44 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57693/
-----------------------------------------------------------

(Updated April 7, 2017, 9:38 a.m.)


Review request for hive, Zoltan Haindrich, Marta Kuczora, Miklos Csanady, Naveen Gangam, Vihang
Karajgaonkar, Yongzhi Chen, and Barna Zsombor Klara.


Bugs: HIVE-16146
    https://issues.apache.org/jira/browse/HIVE-16146


Repository: hive-git


Description
-------

The goal was to generate '\0' markers around the raw log items we want to keep in the golden
files.

To archive this I had to do a small functional change and some small refactor:
- Removed the immutability of the format map in BeeLine, so the test could add the QFile specific
OutputFromat as a possible format
- The PostExecutePrinter and the PreExecutePrinter got a common ancestor, which was due anyway
because PostExecutePrinter reused static methods from PreExecutePrinter. This way I was able
to create QFile specific printers which are generating the desired markers.
- Moved the QFile test to the org.apache.hive.beeline package, so the test classes can use
the package private classes and methods

For one reason or other BeeLine added an extra space character at the end of the lines for
multiline commands - I have removed this space - Will see if this effects any unit test or
not.

With the above mentioned *OutputFromat*, *QFilePreExecutePrinter*, *QFilePostExecutePrinter*
we can mark the lines which are needed in the q.out file, and during the filtering we can
remove the unneeded  parts - I prefer to keep the log level high in the raw files, so in case
of a test failure we can have better understanding of what has happened.

In the test files:
- Updated the beforeExecute, and afterExecute methods to set the new outputformat and the
new hooks
- Removed the query specific filters, since they are non exitstent in the CLI tests
- Simplified the static filterset - currently only contains the filters which are really neccessary
for the actual tests - might grow to the same size than in the QTestUtils - but if we do not
want to run all of the test we would like to keep this list as small as possible
- Removed unnecessary configurations from QFileBuilder which will be not needed in case we
want to mimic the CLI results


Diffs
-----

  beeline/src/java/org/apache/hive/beeline/BeeLine.java 11526a7 
  beeline/src/java/org/apache/hive/beeline/Commands.java 2578728 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d4a6a0 
  itests/src/test/resources/testconfiguration.properties 7a70c9c 
  itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreBeeLineDriver.java 0d63f5d

  itests/util/src/main/java/org/apache/hive/beeline/qfile/QFile.java ae5a349 
  itests/util/src/main/java/org/apache/hive/beeline/qfile/QFileBeeLineClient.java 760fde6

  itests/util/src/main/java/org/apache/hive/beeline/qfile/package-info.java fcd50ec 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java a5c0fcd 
  ql/src/java/org/apache/hadoop/hive/ql/log/LogDivertAppenderForTest.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/session/OperationLog.java c37a633 
  ql/src/test/results/clientpositive/beeline/drop_with_concurrency.q.out 385f9b7 
  ql/src/test/results/clientpositive/beeline/escape_comments.q.out abc0fee 
  ql/src/test/results/clientpositive/beeline/select_dummy_source.q.out PRE-CREATION 
  service/src/java/org/apache/hive/service/cli/operation/OperationManager.java f62ee4e 
  service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 418f453 


Diff: https://reviews.apache.org/r/57693/diff/2/


Testing
-------

Added a new simple query file from CLI driver, and checked that the generated output is the
same


Thanks,

Peter Vary


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message