Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 24 Oct 2017 10:14:00 +0000 (UTC)
From: "Surendra Singh Lilhore (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13095138.1502915723000.43755.1508840040301@Atlassian.JIRA>
In-Reply-To: <JIRA.13095138.1502915723000@Atlassian.JIRA>
References: <JIRA.13095138.1502915723000@Atlassian.JIRA> <JIRA.13095138.1502915723434@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-12310) [SPS]: Provide an option to track
 the status of in progress requests
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 24 Oct 2017 10:14:05 -0000


    [ https://issues.apache.org/jira/browse/HDFS-12310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16216656#comment-16216656 ] 

Surendra Singh Lilhore commented on HDFS-12310:
-----------------------------------------------

Attached V4 patch..

bq. Please fix the comments:
Fixed
bq. It should put spsStatusInfo into the map?
Yes, need to put in map. null check I added only for safety, spsStatusInfo never be null.

bq. The above code is in muitple places, i.e., removeItemTrackInfo, looks suspicious.
{{removeItemTrackInfo}} handling three cases and trying to clean the status from map in all the case. Status will be cleand from map only if it is in SUCCESS status more than 5 minute.
# If trackInfo is directory and inode is null, means start inode is delete but status still there in statusInfo map. We need to clean it, so marking it success.
# If trackInfo is directory and inode not null, check pending work for directory. If all the files processed in directory then mark is success.
# If trackInfo is file then just mark is success becuase all the blocks satisfy the policy.

bq. Would it make sense to put cleanSpsStatus(); into {{ if (!namesystem.isInSafeMode()) }}, and avoid run it for every processed inode.
Fixed. Added check to run {{cleanSpsStatus()}} every 5 minute.

bq. StoragePolicySatisfyPathStatusInfo#setSuccessStatus()/setInProgress() / canRemoveStatus() can be project-private. 
Fixed

bq. Could you add comment to DFSClient# checkStoragePolicySatisfyPathStatus() as well?
Fixed


Please review..

> [SPS]: Provide an option to track the status of in progress requests
> --------------------------------------------------------------------
>
>                 Key: HDFS-12310
>                 URL: https://issues.apache.org/jira/browse/HDFS-12310
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>            Reporter: Uma Maheswara Rao G
>            Assignee: Surendra Singh Lilhore
>         Attachments: HDFS-12310-HDFS-10285-01.patch, HDFS-12310-HDFS-10285-02.patch, HDFS-12310-HDFS-10285-03.patch, HDFS-12310-HDFS-10285-04.patch
>
>
> As per the [~andrew.wang] 's review comments in HDFS-10285, This is the JIRA for tracking about the options how we track the progress of SPS requests.
>  


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org