hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Surendra Singh Lilhore (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12310) [SPS]: Provide an option to track the status of in progress requests
Date Tue, 24 Oct 2017 10:14:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16216656#comment-16216656

Surendra Singh Lilhore commented on HDFS-12310:

Attached V4 patch..

bq. Please fix the comments:
bq. It should put spsStatusInfo into the map?
Yes, need to put in map. null check I added only for safety, spsStatusInfo never be null.

bq. The above code is in muitple places, i.e., removeItemTrackInfo, looks suspicious.
{{removeItemTrackInfo}} handling three cases and trying to clean the status from map in all
the case. Status will be cleand from map only if it is in SUCCESS status more than 5 minute.
# If trackInfo is directory and inode is null, means start inode is delete but status still
there in statusInfo map. We need to clean it, so marking it success.
# If trackInfo is directory and inode not null, check pending work for directory. If all the
files processed in directory then mark is success.
# If trackInfo is file then just mark is success becuase all the blocks satisfy the policy.

bq. Would it make sense to put cleanSpsStatus(); into {{ if (!namesystem.isInSafeMode()) }},
and avoid run it for every processed inode.
Fixed. Added check to run {{cleanSpsStatus()}} every 5 minute.

bq. StoragePolicySatisfyPathStatusInfo#setSuccessStatus()/setInProgress() / canRemoveStatus()
can be project-private. 

bq. Could you add comment to DFSClient# checkStoragePolicySatisfyPathStatus() as well?

Please review..

> [SPS]: Provide an option to track the status of in progress requests
> --------------------------------------------------------------------
>                 Key: HDFS-12310
>                 URL: https://issues.apache.org/jira/browse/HDFS-12310
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>            Reporter: Uma Maheswara Rao G
>            Assignee: Surendra Singh Lilhore
>         Attachments: HDFS-12310-HDFS-10285-01.patch, HDFS-12310-HDFS-10285-02.patch,
HDFS-12310-HDFS-10285-03.patch, HDFS-12310-HDFS-10285-04.patch
> As per the [~andrew.wang] 's review comments in HDFS-10285, This is the JIRA for tracking
about the options how we track the progress of SPS requests.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message