asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Blow <mblow.apa...@gmail.com>
Subject Re: Temporary Datasets
Date Mon, 20 Nov 2017 20:06:39 GMT
>> removing temporary datasets?

+1


On Mon, Nov 20, 2017 at 1:52 PM Murtadha Hubail <hubailmor@gmail.com> wrote:

> Hi all,
>
>
>
> As you might be aware, we have a feature in AsterixDB to create temporary
> datasets that differ from regular datasets in some ways such as:
> Their existence is not persisted in metadata, but only in the CC metadata
> cache.
> They don’t’ generate any transaction logs
> Their files are deleted on NC restart.
> If they are not accessed for some period of time, their metadata records
> are removed from the CC metadata cache.
>
>
> Temporary datasets were originally introduced to serve as a staging area
> between AsterixDB and external systems such as Perglix. However, as the
> system evolved over the years, the assumptions they were built on don’t
> hold anymore and they could lead to undesired consequences such as leaking
> files after a CC restart or inability to access the dataset files on a
> restarted NC. Therefore, I’m proposing to remove the support for the
> current temporary datasets and we may add the feature with a careful design
> at a later stage.
>
>
>
> Any thoughts or concerns on removing temporary datasets?
>
>
>
> Cheers,
>
> Murtadha
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message