Mailing-List: contact dev-help@asterixdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@asterixdb.apache.org
Subject: Re: Temporary Datasets
To: dev@asterixdb.apache.org
References: <F4F7762C-7FD1-4355-B37D-5D19A15CEA79@gmail.com>
 <CAFUTFeaKG0-fYgVNPJJV_dRoJ7avPN8VToY6vzmT-wBUdA2NSw@mail.gmail.com>
 <CAN_YF5xduCuBOXOd-jA7MTE+ioVE3RpYDZkT=+ExtVbpWnFbeQ@mail.gmail.com>
From: Mike Carey <dtabass@gmail.com>
Message-ID: <0c38a7d2-63f5-67f6-fab0-cce7faabab6f@gmail.com>
Date: Mon, 20 Nov 2017 22:16:00 -0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:52.0)
 Gecko/20100101 Thunderbird/52.4.0
MIME-Version: 1.0
In-Reply-To: <CAN_YF5xduCuBOXOd-jA7MTE+ioVE3RpYDZkT=+ExtVbpWnFbeQ@mail.gmail.com>
Content-Type: multipart/alternative;
 boundary="------------CD827C73E976134A45D0E236"
Content-Language: en-US
archived-at: Tue, 21 Nov 2017 06:16:08 -0000

--------------CD827C73E976134A45D0E236
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit

It's only TEMPORARY...  :-)


On 11/20/17 12:41 PM, Ian Maxon wrote:
> Are we going to preserve the keyword?
>
> On Mon, Nov 20, 2017 at 12:06 PM, Michael Blow <mblow.apache@gmail.com> wrote:
>>>> removing temporary datasets?
>> +1
>>
>>
>> On Mon, Nov 20, 2017 at 1:52 PM Murtadha Hubail <hubailmor@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>>
>>>
>>> As you might be aware, we have a feature in AsterixDB to create temporary
>>> datasets that differ from regular datasets in some ways such as:
>>> Their existence is not persisted in metadata, but only in the CC metadata
>>> cache.
>>> They don’t’ generate any transaction logs
>>> Their files are deleted on NC restart.
>>> If they are not accessed for some period of time, their metadata records
>>> are removed from the CC metadata cache.
>>>
>>>
>>> Temporary datasets were originally introduced to serve as a staging area
>>> between AsterixDB and external systems such as Perglix. However, as the
>>> system evolved over the years, the assumptions they were built on don’t
>>> hold anymore and they could lead to undesired consequences such as leaking
>>> files after a CC restart or inability to access the dataset files on a
>>> restarted NC. Therefore, I’m proposing to remove the support for the
>>> current temporary datasets and we may add the feature with a careful design
>>> at a later stage.
>>>
>>>
>>>
>>> Any thoughts or concerns on removing temporary datasets?
>>>
>>>
>>>
>>> Cheers,
>>>
>>> Murtadha
>>>
>>>


--------------CD827C73E976134A45D0E236--