hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Sichi <jsi...@facebook.com>
Subject Meeting notes from the September Hive Contributors Meeting
Date Fri, 17 Sep 2010 19:55:41 GMT
Meeting date:  Sept 13, 2010

Location:  Cloudera Palo Alto office

Attendees:  http://www.meetup.com/Hive-Contributors-Group/calendar/14689507/

Carl Steinbach gave a status update on the 0.6 release.  Since plans for documentation migration
have been deferred to the next release, the only remaining issues are completion of the CREATE
DATABASE feature (HIVE-675), metastore VARCHAR precision widening (HIVE-1364), and metastore
upgrade scripts (HIVE-1427).  HIVE-675 has already been committed to trunk and the backport
for 0.6 is underway.  Carl is still working on companion feature HIVE-1517, but unless it
is done by Sept 17, we'll drop it from the 0.6 release.  Once a build containing the remaining
features passes all unit tests, we'll post a release candidate and vote on it (no additional
acceptance testing is planned for this release).

Next, HIVE-1546 (making semantic analysis pluggable) was discussed.  The Howl team gave an
overview of their use case for reusing Hive DDL processing within Howl's CLI, together with
a description of the roadmap for Howl/Hive project relationship.  Carl raised concerns about
potential dependency creep preventing future Hive refactoring, and Namit Jain proposed reworking
the approach to restrict it to pre/post-analysis hooks (limiting the dependencies exposed)
rather than full-blown analyzer pluggability+subclassing.  It was agreed that the hooks approach
was the best course for balancing all of the concerns and allowing us to achieve the desired
project collaboration benefits.  The Howl team also committed to getting their Howl checkins
going into a public repository as soon as possible, together with setting up continuous integration
to track the health of the combination of Hive+Howl trunks.

Next, HIVE-1609 (partition filtering metastore API) was briefly discussed, and it was agreed
that the Howl team would move the predicate parser from JavaCC to ANTLR and resubmit the patch.

Finally, HIVE-1476 (metastore creating files as service user) was discussed.  It was agreed
that the approach in the proposed patch (performing HDFS operations on the metastore client
side) was a stopgap that we don't really want to include in Hive.  Instead, the correct long-term
solution being developed by Todd Lipcon is to upgrade the Thrift version used by Hive to a
recent one containing his SASL support, and then add impersonation support to the metastore
server.  Since the Howl team's schedule does not allow them to wait for that work to complete
and get tested, they will keep the HIVE-1476 patch privately applied within their own branch
of Hive; it will not be committed on Hive trunk.  Once they are able to move over to the long-term
solution, the stopgap can be discarded.  (In the meantime, we need to work together to minimize
the merge-to-branch impact as the metastore code continues to change on trunk.)

The October meetup will be at Facebook HQ in Palo Alto.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message