hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hive/ViewDev" by JohnSichi
Date Tue, 22 Dec 2009 01:03:25 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hive/ViewDev" page has been changed by JohnSichi.


  With an inheritance model, views and base tables would share a common base class (here called
ColumnSet following the convention in the Common Warehouse Metamodel for lack of a better
- [[Image:Data--Hive--ViewDev--HiveViewInheritance.png]]
+ {{attachment:HiveViewInheritance.png}}
  For a view, most of the storage descriptor (everything other than the column names and types)
would be irrelevant, so this model could be further refined with such discriminations.
@@ -50, +50 @@

  Alternately, if we choose to avoid inheritance, then we could just add a new viewText attribute
to the existing Table class (leaving it null for base tables):
- [[Image:Data--Hive--ViewDev--HiveViewFlat.png]]
+ {{attachment:HiveViewFlat.png}}
  (Storing the view definition as a table property may not work since property values are
limited to VARCHAR(767), and view definitions may be much longer than that, so we'll need
to use a LOB.)
@@ -75, +75 @@

  Metastore object id's can be used for dependency modeling in order to avoid the need to
update dependency records when an object is renamed.  However, we'll need to decide what kinds
of objects can participate in dependencies.  For example, if we restrict it to just tables
and views (and assuming we don't introduce inheritance for views), then we can use a model
like the one below, in which the dependencies are tracked as (supplier,consumer) table pairs.
 (In this model, the TableDependency class is acting as an intersection table for implementing
a many-to-many relationship between suppliers and consumers).
- [[Image:Data--Hive--ViewDev--HiveTableDependency.png]]
+ {{attachment:HiveTableDependency.png}}
  However, if later we want to introduce persistent functions, or track column dependencies,
this model will be insufficient, and we may need to introduce inheritance, with a DependencyParticipant
base class from which tables, columns, functions etc all derive.  (Again, need to verify that
JDO inheritance will actually support what we want here.)

View raw message