hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hive/ViewDev" by JohnSichi
Date Tue, 01 Feb 2011 20:01:16 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hive/ViewDev" page has been changed by JohnSichi.


  === Underlying Partition Dependencies ===
  '''Update 30-Dec-2009''':  Prasad pointed out that even without supporting materialized
views, it may be necessary to provide users with metadata about data dependencies between
views and underlying table partitions so that users can avoid seeing inconsistent results
during the window when not all partitions have been refreshed with the latest data.  One option
is to attempt to derive this information automatically (using an overconservative guess in
cases where the dependency analysis can't be made smart enough); another is to allow view
creators to declare the dependency rules in some fashion as part of the view definition. 
Based on a design review meeting, we will probably go with the automatic analysis approach
once dependency tracking is implemented.  The analysis will be performed on-demand, perhaps
as part of describing the view or submitting a query job against it.  Until this becomes available,
users may be able to do their own analysis either via empirical lineage tools or via view->table
dependency tracking metadata once it is implemented.  See HIVE-1079.
+ '''Update 1-Feb-2011''':  For the latest on this, see [[Hive/PartitionedViews]].
  == Metastore Upgrades ==

View raw message