Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 23 Aug 2017 10:11:00 +0000 (UTC)
From: "Surendra Singh Lilhore (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13090956.1501308911000.100027.1503483060445@Atlassian.JIRA>
In-Reply-To: <JIRA.13090956.1501308911000@Atlassian.JIRA>
References: <JIRA.13090956.1501308911000@Atlassian.JIRA> <JIRA.13090956.1501308911231@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-12225) [SPS]: Optimize extended attributes
 for tracking SPS movements
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 23 Aug 2017 10:11:06 -0000


    [ https://issues.apache.org/jira/browse/HDFS-12225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138187#comment-16138187 ] 

Surendra Singh Lilhore commented on HDFS-12225:
-----------------------------------------------

Thanks [~umamaheswararao] for review..
bq. . Seems like we have 2 kind of classes I can see for tracking the info. 1. ItemInfo for tracking attempted items 2. SatisfyTrackInfo - for tracking storage movement needed items. Should we unify that class naming? How about something like, SatisfyTrackInfo --> ItemInfo and ItemInfo --> AttemptedItemInfo( extends ItemInfo ?) . Does this make sense to you?
Yes, it make sense.

Attached updated patch, please review...

> [SPS]: Optimize extended attributes for tracking SPS movements
> --------------------------------------------------------------
>
>                 Key: HDFS-12225
>                 URL: https://issues.apache.org/jira/browse/HDFS-12225
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>            Reporter: Uma Maheswara Rao G
>            Assignee: Surendra Singh Lilhore
>         Attachments: HDFS-12225-HDFS-10285-01.patch, HDFS-12225-HDFS-10285-02.patch, HDFS-12225-HDFS-10285-03.patch, HDFS-12225-HDFS-10285-04.patch, HDFS-12225-HDFS-10285-05.patch, HDFS-12225-HDFS-10285-06.patch, HDFS-12225-HDFS-10285-07.patch, HDFS-12225-HDFS-10285-08.patch, HDFS-12225-HDFS-10285-09.patch
>
>
> We have discussed to optimize number extended attributes and asked to report separate JIRA while implementing [HDFS-11150 | https://issues.apache.org/jira/browse/HDFS-11150?focusedCommentId=15766127&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15766127]
> This is the JIRA to track that work 
> For the context, comment copied from HDFS-11150
> {quote}
> [~yuanbo] wrote : I've tried that before. There is an issue here if we only mark the directory. When recovering from FsImage, the InodeMap isn't built up, so we don't know the sub-inode of a given inode, in the end, We cannot add these inodes to movement queue in FSDirectory#addToInodeMap, any thoughts?{quote}
> {quote}
> [~umamaheswararao] wrote: I got what you are saying. Ok for simplicity we can add for all Inodes now. For this to handle 100%, we may need intermittent processing, like first we should add them to some intermittentList while loading fsImage, once fully loaded and when starting active services, we should process that list and do required stuff. But it would add some additional complexity may be. Let's do with all file inodes now and we can revisit later if it is really creating issues. How about you raise a JIRA for it and think to optimize separately?
> {quote}
> {quote}
> [~andrew.wang] wrote in HDFS-10285 merge time review comment : HDFS-10899 also the cursor of the iterator in the EZ root xattr to track progress and handle restarts. I wonder if we can do something similar here to avoid having an xattr-per-file being moved.
> {quote}


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org