impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Brown (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3864: qgen: reduce likelihood of create query() exceptions
Date Thu, 21 Jul 2016 16:42:47 GMT
Michael Brown has uploaded a new patch set (#2).

Change subject: IMPALA-3864: qgen: reduce likelihood of create_query() exceptions
......................................................................

IMPALA-3864: qgen: reduce likelihood of create_query() exceptions

1. Fix a bug in which the computation to produce the string for an
   exception was raising a TypeError. We fix the bug by changing how the
   string is built.

2. Fix a bug in which we tried to choose a relational function (defined
   as taking in more than one argument and returning a Boolean) and were
   looking for its weight in QueryProfile.weights.RELATIONAL_FUNCS, but
   the function wasn't defined in that dictionary. We fix the bug by
   defining weights for those functions.

3. Fix a bug in which QueryProfile.choose_func_signatures() was choosing
   a function without taking into account the set of functions in the
   signatures given to it. We fix the bug by pruning off the weights of
   functions that aren't included in provided signatures. We also add a
   note explaining how the weights defined are "best effort", since
   sometimes functions will be pruned.

4. Added Char signatures to LessThan, GreaterThan, LessThanOrEquals,
   GreaterThanOrEquals

5. Misc small changes to aid in debugging or testing:

   a. Added funcs.Signature representation
   b. Increased the number of fake columns when calling
      query_generator.__main__, which is useful for testing.
   c. Renamed a few variables. For some reason in the query_generator
      module there are a lot of overwritten variables, which makes
      debugging difficult.

Testing:

1. impala-python tests/comparison/query_generator.py produces far fewer
   exceptions. The ones for this bug are fixed, but see IMPALA-3890.

2. Full 3 and 6 hour runs of the query generator system don't show any
   obvious regressions in behavior.

Change-Id: Idd9434a92973176aefb99e11e039209cac3cea65
---
M tests/comparison/funcs.py
M tests/comparison/query_generator.py
M tests/comparison/query_profile.py
3 files changed, 49 insertions(+), 18 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/20/3720/2
-- 
To view, visit http://gerrit.cloudera.org:8080/3720
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idd9434a92973176aefb99e11e039209cac3cea65
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Michael Brown <mikeb@cloudera.com>
Gerrit-Reviewer: David Knupp <dknupp@cloudera.com>
Gerrit-Reviewer: Taras Bobrovytsky <tbobrovytsky@cloudera.com>

Mime
View raw message