hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Casey Brotherton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8118) Delay in checkpointing Trash can leave trash for 2 intervals before deleting
Date Thu, 06 Aug 2015 16:05:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660213#comment-14660213
] 

Casey Brotherton commented on HDFS-8118:
----------------------------------------

Talked with Harsh about this offline.

Still working through a testcase.  We have talked about a method for delaying between the
different checkpoints.
There are still difficulties, though , as the problem occurs when two different users delete
files, and there is a delay between checkpointing each user's deletes.

However, TestTrash operates under the assumption that there is only one user that is creating
files, and removing them for the trash.
( For example, the shell is used to getCurrentTrashDirectory, and that will only return one
location.  For a test, I will either need to 
break Object Oriented walls, and create a path with users/d/.Trash/Current, or create a test
that doesn't really test for the issue. )

Still trying to work on this.


> Delay in checkpointing Trash can leave trash for 2 intervals before deleting
> ----------------------------------------------------------------------------
>
>                 Key: HDFS-8118
>                 URL: https://issues.apache.org/jira/browse/HDFS-8118
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Casey Brotherton
>            Assignee: Casey Brotherton
>            Priority: Trivial
>         Attachments: HDFS-8118.patch
>
>
> When the fs.trash.checkpoint.interval and the fs.trash.interval are set non-zero and
the same, it is possible for trash to be left for two intervals.
> The TrashPolicyDefault will use a floor and ceiling function to ensure that the Trash
will be checkpointed every "interval" of minutes.
> Each user's trash is checkpointed individually.  The time resolution of the checkpoint
timestamp is to the second.
> If the seconds switch while one user is checkpointing, then the next user's timestamp
will be later.
> This will cause the next user's checkpoint to not be deleted at the next interval.
> I have recreated this in a lab cluster 
> I also have a suggestion for a patch that I can upload later tonight after testing it
further.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message