asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abdullah alamoudi <bamou...@gmail.com>
Subject Re: Temporary Datasets
Date Tue, 21 Nov 2017 01:34:22 GMT
+1

> On Nov 20, 2017, at 12:06 PM, Michael Blow <mblow.apache@gmail.com> wrote:
> 
>>> removing temporary datasets?
> 
> +1
> 
> 
> On Mon, Nov 20, 2017 at 1:52 PM Murtadha Hubail <hubailmor@gmail.com> wrote:
> 
>> Hi all,
>> 
>> 
>> 
>> As you might be aware, we have a feature in AsterixDB to create temporary
>> datasets that differ from regular datasets in some ways such as:
>> Their existence is not persisted in metadata, but only in the CC metadata
>> cache.
>> They don’t’ generate any transaction logs
>> Their files are deleted on NC restart.
>> If they are not accessed for some period of time, their metadata records
>> are removed from the CC metadata cache.
>> 
>> 
>> Temporary datasets were originally introduced to serve as a staging area
>> between AsterixDB and external systems such as Perglix. However, as the
>> system evolved over the years, the assumptions they were built on don’t
>> hold anymore and they could lead to undesired consequences such as leaking
>> files after a CC restart or inability to access the dataset files on a
>> restarted NC. Therefore, I’m proposing to remove the support for the
>> current temporary datasets and we may add the feature with a careful design
>> at a later stage.
>> 
>> 
>> 
>> Any thoughts or concerns on removing temporary datasets?
>> 
>> 
>> 
>> Cheers,
>> 
>> Murtadha
>> 
>> 


Mime
View raw message