hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergio Pena <>
Subject Re: Review Request 48233: HIVE-13884: Disallow queries fetching more than a configured number of partitions in PartitionPruner
Date Mon, 13 Jun 2016 18:28:14 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated June 13, 2016, 6:28 p.m.)

Review request for hive, Mohit Sabharwal and Naveen Gangam.


Addressed feedback changes from Kapil and Reuben.

Bugs: HIVE-13884

Repository: hive-git


The patch verifies the # of partitions a table has before fetching any from the metastore.
t checks that limit from 'hive.limit.query.max.table.partition'.

A limitation added here is that the variable must be on hive-site.xml in order to work, and
it does not accept to set this through beeline because does not read the
variables set through beeline. I think it is better to keep it this way to avoid users changing
the value on fly, and crashing the metastore.

Another change is that EXPLAIN commands won't be executed either. EXPLAIN commands need to
fetch partitions in order to create the operator tree. If we allow EXPLAIN to do that, then
we may have the same OOM situations for large partitions.

Diffs (updated)

  metastore/src/java/org/apache/hadoop/hive/metastore/ c0827ea9d47e569d9697649a7e16d196de3de14d

  metastore/src/java/org/apache/hadoop/hive/metastore/ c135179b97354108f842a5ca2de0c6f0ef28b7fc

  metastore/src/java/org/apache/hadoop/hive/metastore/ f98de1326956b19b9d28fc9b1fcdede8d851180d

  metastore/src/java/org/apache/hadoop/hive/metastore/ a6d3f5385b33b8a4e31ee20ca5cb8f58c97c8702

  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/ 31f0d7b89670b8a749bbe8a7ff2b4ff9f059a8e2

  metastore/src/test/org/apache/hadoop/hive/metastore/ 3152e77c3c7152ac4dbe7e779ce35f28044fe3c9

  metastore/src/test/org/apache/hadoop/hive/metastore/ 86a243609b23e2ca9bb8849f0da863a95e477d5c

  ql/src/java/org/apache/hadoop/hive/ql/parse/ cd3c86064df3e7febcc16e03aab6ce407e0dc8a0



Waiting for HiveQA.


Sergio Pena

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message