Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <20151123200035.GR4878@tpx>
References: 
 <CAC15qqXiFk-giNuey0EHQfoo2b9tAg9DSuVdVYg8Yr=GvhExdw@mail.gmail.com>
 <20151123200035.GR4878@tpx>
From: Dmitriy Setrakyan <dsetrakyan@apache.org>
Date: Tue, 24 Nov 2015 02:52:11 +0300
Message-ID: 
 <CA+0=VoVv-mSY_qGGEG9n20XKPH4fDME7BhnWf16WmjRybNiBCA@mail.gmail.com>
Subject: Re: Igfs PURGE events: do we need them?
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary=047d7b417ff944172e05253dec58

--047d7b417ff944172e05253dec58
Content-Type: text/plain; charset=UTF-8

Cos,

The main reason soft delete was added is performance. Without soft-delete,
the delete operation would have to wait until a file is fully deleted from
a folder, which may take time.

As far as secondary FS handling it, IGFS does not require a secondary FS,
so we should account for cases when IGFS is running stand-alone.

Thoughts?

D.

On Mon, Nov 23, 2015 at 11:00 PM, Konstantin Boudnik <cos@apache.org> wrote:

> Let me ask a different question: what's the point of having the concept of
> TRASH?
>
> Here's an example why I think the 'soft' delete would only complicate
> thing.
> Suppose IGFS is sitting on top of HDFS and both have 'Trash' enabled. Now,
> the file is getting soft-deleted from IGFS and is moved to TRASH folder.
> But
> in HDFS it is also a move to a place which doesn't have any special meaning
> for HDFS.
>
> Even worst, if IFGS TRASH is linked to HDFS .Trash. HDFS has it's own
> policy
> on how to clean that up, which is likely to be different from that on IGFS.
> Often enough, HDFS .Trash is simply disabled. This discrepancy is going to
> create a situation when a file should still be in TRASH, but the secondary
> FS
> has already purged it.
>
> And what if yet another secondary file system like S3 has yet another
> policy
> around their own trash, which they don't even have, I believe?
>
> Where I am going with this is pretty straight forward: let's drop the
> soft-delete support and let the secondary FS to deal with it. If there's no
> secondary FS configured - the content of deleted file will have to
> retrieved
> by other means.
>
> Thoughts?
>   Cos
>
> On Fri, Nov 20, 2015 at 07:15PM, Ivan V. wrote:
> > Hi, dev,
> > need opinions on the question discussed in
> > https://issues.apache.org/jira/browse/IGNITE-1679  (IGFS: Purge event is
> > inconsistent).
> > In short: in Igfs we have "soft" delete that moves the deleted file or
> > folder to special "TRASH" folder.
> > Special async worker walks inside TRASH and removes the items
> permanently.
> > When an item is completely removed, an event of type
> > org.apache.ignite.events.EventType#EVT_IGFS_FILE_PURGED  is fired.
> > But such events are now fired only for files, and only in case if such
> file
> > was deleted itself, but not a part of a folder sub-tree. It's quite
> obvious
> > that such behavior is not quite consistent, so we should either get rid
> of
> > PURGE events at all, or make them consistent.
> > In the latter case it would be good to have answer to the question: what
> > are real  use cases when we may need the purge events ? (Now they seem to
> > be used in tests only).
> > If we don't have such real use cases, are there any objections to get rid
> > of the purge events at all?
> > Thanks in advance.
>

--047d7b417ff944172e05253dec58--