hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nandakumar Vadivelu <nvadiv...@hortonworks.com>
Subject Re: 回复:[VOTE] Merge Storage Policy Satisfier (SPS) [HDFS-10285] feature branch to trunk
Date Tue, 07 Aug 2018 14:47:12 GMT
+1 for the merge. Thanks for the work Uma et al.

-Nanda

On 8/7/18, 7:35 PM, "Vinayakumar B" <vinayakumarb@apache.org> wrote:

    +1
    
    Great work guys.
    -Vinay
    
    On Tue, 7 Aug 2018, 7:20 pm Anu Engineer, <aengineer@hortonworks.com> wrote:
    
    > +1, Sorry for the late vote. Thanks for the perseverance and seeing this
    > thru.
    >
    > --Anu
    >
    >
    > On 8/7/18, 1:04 AM, "郑锴(铁杰)" <zhengkai.zk@alibaba-inc.com> wrote:
    >
    >     +1 for the work to be in. Thanks Uma and folks for the hard taking!
    >
    >     When it's in, I'd suggest we use a more general name for the new
    > daemon service. It'd be good to evolve and support more self-running admin
    > functionalities incubated first there before doing it directly in NameNode.
    >
    >     Regards,
    >     Kai
    >     ------------------------------------------------------------------
    >     发件人:Uma Maheswara Rao G <hadoop.uma@gmail.com>
    >     发送时间:2018年8月1日(星期三) 14:38
    >     收件人:hdfs-dev <hdfs-dev@hadoop.apache.org>
    >     主 题:[VOTE] Merge Storage Policy Satisfier (SPS) [HDFS-10285] feature
    > branch to trunk
    >
    >     Hi All,
    >
    >
    >
    >      From the positive responses from JIRA discussion and no objections
    > from
    >     below DISCUSS thread [1], I am converting it to voting thread.
    >
    >
    >
    >      Last couple of weeks we spent time on testing the feature and so far
    > it is
    >     working fine. Surendra uploaded a test report at HDFS-10285:  [2]
    >
    >
    >
    >      In this phase, we provide to run SPS outside of Namenode only and as a
    >     next phase we continue to discuss and work on to enable it as Internal
    > SPS
    >     as explained below. We have got clean QA report on branch and if there
    > are
    >     any static tool comments triggered later while running this thread, we
    > will
    >     make sure to fix them before merge. We committed and continue to
    > improve
    >     the code on trunk. Please refer to HDFS-10285 for discussion details.
    >
    >
    >
    >      This has been a long effort and we're grateful for the support we've
    >     received from the community. In particular, thanks to Andrew Wang,
    > Anoop
    >     Sam John, Anu Engineer, Chris Douglas, Daryn Sharp, Du Jingcheng , Ewan
    >     Higgs, Jing Zhao, Kai Zheng,  Rakesh R, Ramkrishna , Surendra Singh
    > Lilhore
    >     , Thomas Demoor, Uma Maheswara Rao G, Vinayakumar, Virajith,  Wei Zhou,
    >     Yuanbo Liu. Without these members effort, this feature might not have
    >     reached to this state.
    >
    >
    >
    >     To start with, here is my +1
    >
    >     It will end on 6th Aug.
    >
    >
    >
    >     Regards,
    >
    >     Uma
    >
    >     [1]  https://s.apache.org/bhyu
    >     [2]  https://s.apache.org/AXvL
    >
    >
    >     On Wed, Jun 27, 2018 at 3:21 PM, Uma Maheswara Rao G <
    > hadoop.uma@gmail.com>
    >     wrote:
    >
    >     > Hi All,
    >     >
    >     >   After long discussions(offline and on JIRA) on SPS, we came to a
    >     > conclusion on JIRA(HDFS-10285) that, we will go ahead with External
    > SPS
    >     > merge in first phase. In this phase process will not be running
    > inside
    >     > Namenode.
    >     >   We will continue discussion on Internal SPS. Current code base
    > supports
    >     > both internal and external option. We have review comments for
    > Internal
    >     > which needs some additional works for analysis and testing etc. We
    > will
    >     > move Internal SPS work to under HDFS-12226 (Follow-on work for SPS
    > in NN)
    >     > We are working on cleanup task HDFS-13076 for the merge. .
    >     > For more clarity on Internal and External SPS proposal thoughts,
    > please
    >     > refer to JIRA HDFS-10285.
    >     >
    >     > If there are no objections with this, I will go ahead for voting
    > soon.
    >     >
    >     > Regards,
    >     > Uma
    >     >
    >     > On Fri, Nov 17, 2017 at 3:16 PM, Uma Maheswara Rao G <
    > hadoop.uma@gmail.com
    >     > > wrote:
    >     >
    >     >> Update: We worked on the review comments and additional JIRAs above
    >     >> mentioned.
    >     >>
    >     >> >1. After the feedbacks from Andrew, Eddy, Xiao in JIRA reviews,
we
    >     >> planned to take up the support for recursive API support.
    > HDFS-12291<
    >     >> https://issues.apache.org/jira/browse/HDFS-12291>
    >     >>
    >     >> We provided the recursive API support now.
    >     >>
    >     >> >2. Xattr optimizations HDFS-12225<https://issues.apac
    >     >> he.org/jira/browse/HDFS-12225>
    >     >> Improved this portion as well
    >     >>
    >     >> >3. Few other review comments already fixed and committed
    > HDFS-12214<
    >     >> https://issues.apache.org/jira/browse/HDFS-12214>
    >     >> Fixed the comments.
    >     >>
    >     >> We are continuing to test the feature and working so far well. Also
    > we
    >     >> uploaded a combined patch and got the good QA report.
    >     >>
    >     >> If there are no further objections, we would like to go for merge
    > vote
    >     >> tomorrow. Please by default this feature will be disabled.
    >     >>
    >     >> Regards,
    >     >> Uma
    >     >>
    >     >> On Fri, Aug 18, 2017 at 11:27 PM, Gangumalla, Uma <
    >     >> uma.gangumalla@intel.com> wrote:
    >     >>
    >     >>> Hi Andrew,
    >     >>>
    >     >>> >Great to hear. It'd be nice to define which use cases are met
by
    > the
    >     >>> current version of SPS, and which will be handled after the merge.
    >     >>> After the discussions in JIRA, we planned to support recursive API
    > as
    >     >>> well. The primary use cases we planned was for Hbase. Please check
    > next
    >     >>> point for use case details.
    >     >>>
    >     >>> >A bit more detail in the design doc on how HBase would use this
    > feature
    >     >>> would also be helpful. Is there an HBase JIRA already?
    >     >>> Please find the usecase details at this comment in JIRA:
    >     >>> https://issues.apache.org/jira/browse/HDFS-10285?focusedComm
    >     >>> entId=16120227&page=com.atlassian.jira.plugin.system.issueta
    >     >>> bpanels:comment-tabpanel#comment-16120227
    >     >>>
    >     >>> >I also spent some more time with the design doc and posted a
few
    >     >>> questions on the JIRA.
    >     >>> Thank you for the reviews.
    >     >>>
    >     >>> To summarize the discussions in JIRA:
    >     >>> 1. After the feedbacks from Andrew, Eddy, Xiao in JIRA reviews,
we
    >     >>> planned to take up the support for recursive API support.
    > HDFS-12291<
    >     >>> https://issues.apache.org/jira/browse/HDFS-12291> (Rakesh started
    > the
    >     >>> work on it)
    >     >>> 2. Xattr optimizations HDFS-12225<https://issues.apac
    >     >>> he.org/jira/browse/HDFS-12225> (Patch available)
    >     >>> 3. Few other review comments already fixed and committed
    > HDFS-12214<
    >     >>> https://issues.apache.org/jira/browse/HDFS-12214>
    >     >>>
    >     >>> For tracking the follow-up tasks we filed JIRA HDFS-12226, they
    > should
    >     >>> not be critical for merge.
    >     >>>
    >     >>> Regards,
    >     >>> Uma
    >     >>>
    >     >>> From: Andrew Wang <andrew.wang@cloudera.com<mailto:
    >     >>> andrew.wang@cloudera.com>>
    >     >>> Date: Friday, July 28, 2017 at 11:33 AM
    >     >>> To: Uma Gangumalla <uma.gangumalla@intel.com<mailto:
    >     >>> uma.gangumalla@intel.com>>
    >     >>> Cc: "hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>"
    > <
    >     >>> hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>>
    >     >>> Subject: Re: [DISCUSS] Merge Storage Policy Satisfier (SPS)
    > [HDFS-10285]
    >     >>> feature branch to trunk
    >     >>>
    >     >>> Hi Uma,
    >     >>>
    >     >>> > If there are still plans to make changes that affect
    > compatibility
    >     >>> (the hybrid RPC and bulk DN work mentioned sound like they would),
    > then we
    >     >>> can cut branch-3 first, or wait to merge until after these tasks
    > are
    >     >>> finished.
    >     >>> [Uma] We don’t see that 2 items as high priority for the feature.
    > Users
    >     >>> would be able to use the feature with current code base and API.
    > So, we
    >     >>> would consider them after branch-3 only. That should be perfectly
    > fine IMO.
    >     >>> The current API is very much useful for Hbase scenario. In Hbase
    > case, they
    >     >>> will rename files under to different policy directory. They will
    > not set
    >     >>> the policies always. So, when rename files under to different
    > policy
    >     >>> directory, they can simply call satisfyStoragePolicy, they don’t
    > need any
    >     >>> hybrid API.
    >     >>>
    >     >>> Great to hear. It'd be nice to define which usecases are met by
the
    >     >>> current version of SPS, and which will be handled after the merge.
    >     >>>
    >     >>> A bit more detail in the design doc on how HBase would use this
    > feature
    >     >>> would also be helpful. Is there an HBase JIRA already?
    >     >>>
    >     >>> I also spent some more time with the design doc and posted a few
    >     >>> questions on the JIRA.
    >     >>>
    >     >>> Best,
    >     >>> Andrew
    >     >>>
    >     >>
    >     >>
    >     >
    >
    >
    >
    >
    > ---------------------------------------------------------------------
    > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
    > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
    >
    

Mime
View raw message