atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shwetha G S (JIRA)" <>
Subject [jira] [Created] (ATLAS-442) Hive model to handle cascaded deletes
Date Tue, 19 Jan 2016 05:42:39 GMT
Shwetha G S created ATLAS-442:

             Summary: Hive model to handle cascaded deletes
                 Key: ATLAS-442
             Project: Atlas
          Issue Type: Bug
            Reporter: Shwetha G S

Ideally, hive model should be like this:
type - attributes
hive_table - ..., partitions(list<hive_partition>), ...
hive_partition - values(list<string>), ...

partitions attribute in hive_table is composite. Add partition is frequent in hive, and this
would result in create new partition and add this reference to table. So, this results in
create entity + update array for hive_table entity. Since the number of partitions can be
large, we want to avoid updates to hive_table entity every time. 

Hence hive model is implemented now as:
hive_table - ...
hive_partition - table(hive_table), values(list<string>), ...

In this model, there is no way to indicate that partitions are composite attributes of table
and hence deleting table will nor result in deleting of partitions. But we need the cascading
delete of partitions in this case. We need to figure out a way to solve this.

This message was sent by Atlassian JIRA

View raw message