hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <>
Subject [jira] [Commented] (HIVE-4914) filtering via partition name should be done inside metastore server (implementation)
Date Fri, 30 Aug 2013 13:30:52 GMT


Phabricator commented on HIVE-4914:

ashutoshc has requested changes to the revision "HIVE-4914 [jira] filtering via partition
name should be done inside metastore server (implementation)".


  metastore/if/hive_metastore.thrift:282 Can you add a comment here defining this boolean?
  metastore/if/hive_metastore.thrift:510 Instead of list of parameters, can you define a struct
which is passed in as an argument. That way in future if we need to add another parameter
for this function, it will still be back-compat.
  metastore/src/java/org/apache/hadoop/hive/metastore/ Consider using
MetaStoreUtils::newInstance() for this.
  metastore/src/java/org/apache/hadoop/hive/metastore/ Can you add a comment
here saying this class is created via reflection to avoid circular dependency on ql package?
  metastore/if/hive_metastore.thrift:281  Set<Partition> instead of list<Partition>
  ql/src/java/org/apache/hadoop/hive/ql/exec/ Why not enhance existing deserializeExpressions()
to allow it to throw exception? Or, atleast reuse the common code.
  metastore/src/java/org/apache/hadoop/hive/metastore/parser/ Add description
of @param FilterBuilder in javadoc here.
  metastore/src/java/org/apache/hadoop/hive/metastore/parser/ Update
javadoc with new param.
  metastore/src/java/org/apache/hadoop/hive/metastore/parser/ Update
  metastore/src/java/org/apache/hadoop/hive/metastore/parser/ ExpressionTree
is getting too large. Better to put this class and FilterBuilder in another file?
  metastore/src/java/org/apache/hadoop/hive/metastore/ With query.setRange()
this is no longer required.
  metastore/src/java/org/apache/hadoop/hive/metastore/ Didn't get this.
With this patch, Hive client wont do any work, right?
  metastore/src/java/org/apache/hadoop/hive/metastore/ This TODO is important
to resolve. Can you follow up on this?
  metastore/src/java/org/apache/hadoop/hive/metastore/ Is this just for
tests? Or is it needed? Either way, can you add a comment for it.
  metastore/src/java/org/apache/hadoop/hive/metastore/ This class is
getting too large. May be a good idea to put some of the helper inner classes and methods
in MetaStoreUtils class.
  metastore/src/java/org/apache/hadoop/hive/metastore/ This class
is also growing in size. Probably, put it in a seprate file along with TreeVisitor.




To: JIRA, ashutoshc, sershe

> filtering via partition name should be done inside metastore server (implementation)
> ------------------------------------------------------------------------------------
>                 Key: HIVE-4914
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-4914.01.patch, HIVE-4914.D12561.1.patch, HIVE-4914-only-no-gen.patch,
HIVE-4914-only.patch, HIVE-4914.patch, HIVE-4914.patch, HIVE-4914.patch
> Currently, if the filter pushdown is impossible (which is most cases), the client gets
all partition names from metastore, filters them, and asks for partitions by names for the
filtered set.
> Metastore server code should do that instead; it should check if pushdown is possible
and do it if so; otherwise it should do name-based filtering.
> Saves the roundtrip with all partition names from the server to client, and also removes
the need to have pushdown viability checking on both sides.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message