asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Carey <dtab...@gmail.com>
Subject Re: Temporary Datasets
Date Tue, 21 Nov 2017 06:16:00 GMT
It's only TEMPORARY...  :-)


On 11/20/17 12:41 PM, Ian Maxon wrote:
> Are we going to preserve the keyword?
>
> On Mon, Nov 20, 2017 at 12:06 PM, Michael Blow <mblow.apache@gmail.com> wrote:
>>>> removing temporary datasets?
>> +1
>>
>>
>> On Mon, Nov 20, 2017 at 1:52 PM Murtadha Hubail <hubailmor@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>>
>>>
>>> As you might be aware, we have a feature in AsterixDB to create temporary
>>> datasets that differ from regular datasets in some ways such as:
>>> Their existence is not persisted in metadata, but only in the CC metadata
>>> cache.
>>> They don’t’ generate any transaction logs
>>> Their files are deleted on NC restart.
>>> If they are not accessed for some period of time, their metadata records
>>> are removed from the CC metadata cache.
>>>
>>>
>>> Temporary datasets were originally introduced to serve as a staging area
>>> between AsterixDB and external systems such as Perglix. However, as the
>>> system evolved over the years, the assumptions they were built on don’t
>>> hold anymore and they could lead to undesired consequences such as leaking
>>> files after a CC restart or inability to access the dataset files on a
>>> restarted NC. Therefore, I’m proposing to remove the support for the
>>> current temporary datasets and we may add the feature with a careful design
>>> at a later stage.
>>>
>>>
>>>
>>> Any thoughts or concerns on removing temporary datasets?
>>>
>>>
>>>
>>> Cheers,
>>>
>>> Murtadha
>>>
>>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message