Mailing-List: contact dev-help@asterixdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@asterixdb.apache.org
Subject: Re: Metadata Dependencies
To: dev@asterixdb.apache.org
References: <CAE1xAnvTVASfgGf4CC283Nz7d3wqf7hTHUGe06fzLE-9c=5cgg@mail.gmail.com>
 <accd7d01-a4a1-7030-0b0c-54788bde3e2d@gmail.com>
 <CAE1xAnu3KBxbn7HBeKrfuzNaEAPDcgrxE0_1tS2ngv+inTCUZQ@mail.gmail.com>
 <5a9a1868-fe17-7c04-a236-db5f4f8112ac@gmail.com>
 <CAE1xAnv+j9-Ck8Lqk-56nfwOgJUQDVCdpNe==_WBd6191bYE1g@mail.gmail.com>
From: Mike Carey <dtabass@gmail.com>
Message-ID: <e34ace6a-91cb-25f3-9c59-c47f42824661@gmail.com>
Date: Thu, 13 Oct 2016 12:14:57 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0)
 Gecko/20100101 Thunderbird/45.4.0
MIME-Version: 1.0
In-Reply-To: <CAE1xAnv+j9-Ck8Lqk-56nfwOgJUQDVCdpNe==_WBd6191bYE1g@mail.gmail.com>
Content-Type: multipart/alternative;
 boundary="------------76D3F89AA15628E205CB9C11"
archived-at: Thu, 13 Oct 2016 19:15:08 -0000

--------------76D3F89AA15628E205CB9C11
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Do we absolutely need that fine level of granularity?  (Maybe a few 
examples would help.)


On 10/13/16 11:40 AM, Steven Jacobs wrote:
> I lean towards B as well. The interesting question is how to succinctly
> store the dependency field information in this catalog, i.e. I have these
> fields which correspond to the primary key fields of this other metadata
> dataset.
> Steven
>
> On Wednesday, October 12, 2016, Mike Carey <dtabass@gmail.com> wrote:
>
>> Got it.  IMO option B) is probably "better" in the sense that it seems
>> "safer" and "more transparent".  With A) not much might be known about why
>> those catalog entries are there, assuming a generic notion of "entity", and
>> there's always the potential for a buggy extension to forget to delete a
>> dependency and therefore force the non-deletion of a given dataverse.  With
>> B) you know what the issue is - the supposed existence of a metadata
>> dataset - and you have the possible recourse of verifying the existence of
>> such a dataset.  (E.g., if a buggy extension drops its dataset but forgets
>> to update this registry, you can at least discover manually that the
>> dataset is already gone.)  Maybe not a big deal, but....
>>
>>
>> On 10/12/16 3:11 PM, Steven Jacobs wrote:
>>
>>> Yes, "catalog" would be the better word here. Thanks for the
>>> clarification.
>>>
>>> Steven
>>>
>>> On Wed, Oct 12, 2016 at 12:38 PM, Mike Carey <dtabass@gmail.com> wrote:
>>>
>>> I assume that by "index" you really mean "catalog" (as opposed to index in
>>>> the physical sense)?  I.e., your two proposals are about a possible
>>>> additional metadata dataset in support of extension dependency
>>>> management?
>>>>
>>>>
>>>> On 10/12/16 9:47 AM, Steven Jacobs wrote:
>>>>
>>>> Hi,
>>>>> I am trying to get feedback from the group in general about how best to
>>>>> add
>>>>> Metadata Dependencies, particularly in the context of extensions. The
>>>>> functionality that is needed is as follows:
>>>>>
>>>>> *When dropping a metadata entity A (e.g. a dataverse), first check
>>>>> whether
>>>>> A has any metadata entities that depend on it. Then allow these entities
>>>>> two options:*
>>>>>
>>>>> *1) block: don't drop A while this dependent entity exists*
>>>>> *2) chain: when dropping A, drop this dependent entity as well.*
>>>>>
>>>>> One issue to keep in mind here is that with extensions, Asterix will not
>>>>> currently know about metadata datasets added by extensions.
>>>>>
>>>>> So far we have two proposals for this issue:
>>>>>
>>>>> A) Metadata dependency index. When you create an entity that depends on
>>>>> another, add an entry to this index. Check this index when dropping an
>>>>> entity.
>>>>>
>>>>> B) Metadata data set index. All Metadata datasets would be registered in
>>>>> this index, and then we can query them as needed when dropping an
>>>>> entity.
>>>>> In this case we would need some way to specify which fields in these
>>>>> datasets represent dependencies in order to query them properly.
>>>>>
>>>>> We would like any feedback on these two solutions or proposals for
>>>>> alternate solutions.
>>>>>
>>>>> Steven
>>>>>
>>>>>
>>>>>


--------------76D3F89AA15628E205CB9C11--