ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: Igfs PURGE events: do we need them?
Date Mon, 23 Nov 2015 23:52:11 GMT

The main reason soft delete was added is performance. Without soft-delete,
the delete operation would have to wait until a file is fully deleted from
a folder, which may take time.

As far as secondary FS handling it, IGFS does not require a secondary FS,
so we should account for cases when IGFS is running stand-alone.



On Mon, Nov 23, 2015 at 11:00 PM, Konstantin Boudnik <cos@apache.org> wrote:

> Let me ask a different question: what's the point of having the concept of
> Here's an example why I think the 'soft' delete would only complicate
> thing.
> Suppose IGFS is sitting on top of HDFS and both have 'Trash' enabled. Now,
> the file is getting soft-deleted from IGFS and is moved to TRASH folder.
> But
> in HDFS it is also a move to a place which doesn't have any special meaning
> for HDFS.
> Even worst, if IFGS TRASH is linked to HDFS .Trash. HDFS has it's own
> policy
> on how to clean that up, which is likely to be different from that on IGFS.
> Often enough, HDFS .Trash is simply disabled. This discrepancy is going to
> create a situation when a file should still be in TRASH, but the secondary
> FS
> has already purged it.
> And what if yet another secondary file system like S3 has yet another
> policy
> around their own trash, which they don't even have, I believe?
> Where I am going with this is pretty straight forward: let's drop the
> soft-delete support and let the secondary FS to deal with it. If there's no
> secondary FS configured - the content of deleted file will have to
> retrieved
> by other means.
> Thoughts?
>   Cos
> On Fri, Nov 20, 2015 at 07:15PM, Ivan V. wrote:
> > Hi, dev,
> > need opinions on the question discussed in
> > https://issues.apache.org/jira/browse/IGNITE-1679  (IGFS: Purge event is
> > inconsistent).
> > In short: in Igfs we have "soft" delete that moves the deleted file or
> > folder to special "TRASH" folder.
> > Special async worker walks inside TRASH and removes the items
> permanently.
> > When an item is completely removed, an event of type
> > org.apache.ignite.events.EventType#EVT_IGFS_FILE_PURGED  is fired.
> > But such events are now fired only for files, and only in case if such
> file
> > was deleted itself, but not a part of a folder sub-tree. It's quite
> obvious
> > that such behavior is not quite consistent, so we should either get rid
> of
> > PURGE events at all, or make them consistent.
> > In the latter case it would be good to have answer to the question: what
> > are real  use cases when we may need the purge events ? (Now they seem to
> > be used in tests only).
> > If we don't have such real use cases, are there any objections to get rid
> > of the purge events at all?
> > Thanks in advance.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message