hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinayakumar B <vinayakum...@apache.org>
Subject Re: 回复:[VOTE] Merge Storage Policy Satisfier (SPS) [HDFS-10285] feature branch to trunk
Date Tue, 07 Aug 2018 14:04:57 GMT
+1

Great work guys.
-Vinay

On Tue, 7 Aug 2018, 7:20 pm Anu Engineer, <aengineer@hortonworks.com> wrote:

> +1, Sorry for the late vote. Thanks for the perseverance and seeing this
> thru.
>
> --Anu
>
>
> On 8/7/18, 1:04 AM, "郑锴(铁杰)" <zhengkai.zk@alibaba-inc.com> wrote:
>
>     +1 for the work to be in. Thanks Uma and folks for the hard taking!
>
>     When it's in, I'd suggest we use a more general name for the new
> daemon service. It'd be good to evolve and support more self-running admin
> functionalities incubated first there before doing it directly in NameNode.
>
>     Regards,
>     Kai
>     ------------------------------------------------------------------
>     发件人:Uma Maheswara Rao G <hadoop.uma@gmail.com>
>     发送时间:2018年8月1日(星期三) 14:38
>     收件人:hdfs-dev <hdfs-dev@hadoop.apache.org>
>     主 题:[VOTE] Merge Storage Policy Satisfier (SPS) [HDFS-10285] feature
> branch to trunk
>
>     Hi All,
>
>
>
>      From the positive responses from JIRA discussion and no objections
> from
>     below DISCUSS thread [1], I am converting it to voting thread.
>
>
>
>      Last couple of weeks we spent time on testing the feature and so far
> it is
>     working fine. Surendra uploaded a test report at HDFS-10285:  [2]
>
>
>
>      In this phase, we provide to run SPS outside of Namenode only and as a
>     next phase we continue to discuss and work on to enable it as Internal
> SPS
>     as explained below. We have got clean QA report on branch and if there
> are
>     any static tool comments triggered later while running this thread, we
> will
>     make sure to fix them before merge. We committed and continue to
> improve
>     the code on trunk. Please refer to HDFS-10285 for discussion details.
>
>
>
>      This has been a long effort and we're grateful for the support we've
>     received from the community. In particular, thanks to Andrew Wang,
> Anoop
>     Sam John, Anu Engineer, Chris Douglas, Daryn Sharp, Du Jingcheng , Ewan
>     Higgs, Jing Zhao, Kai Zheng,  Rakesh R, Ramkrishna , Surendra Singh
> Lilhore
>     , Thomas Demoor, Uma Maheswara Rao G, Vinayakumar, Virajith,  Wei Zhou,
>     Yuanbo Liu. Without these members effort, this feature might not have
>     reached to this state.
>
>
>
>     To start with, here is my +1
>
>     It will end on 6th Aug.
>
>
>
>     Regards,
>
>     Uma
>
>     [1]  https://s.apache.org/bhyu
>     [2]  https://s.apache.org/AXvL
>
>
>     On Wed, Jun 27, 2018 at 3:21 PM, Uma Maheswara Rao G <
> hadoop.uma@gmail.com>
>     wrote:
>
>     > Hi All,
>     >
>     >   After long discussions(offline and on JIRA) on SPS, we came to a
>     > conclusion on JIRA(HDFS-10285) that, we will go ahead with External
> SPS
>     > merge in first phase. In this phase process will not be running
> inside
>     > Namenode.
>     >   We will continue discussion on Internal SPS. Current code base
> supports
>     > both internal and external option. We have review comments for
> Internal
>     > which needs some additional works for analysis and testing etc. We
> will
>     > move Internal SPS work to under HDFS-12226 (Follow-on work for SPS
> in NN)
>     > We are working on cleanup task HDFS-13076 for the merge. .
>     > For more clarity on Internal and External SPS proposal thoughts,
> please
>     > refer to JIRA HDFS-10285.
>     >
>     > If there are no objections with this, I will go ahead for voting
> soon.
>     >
>     > Regards,
>     > Uma
>     >
>     > On Fri, Nov 17, 2017 at 3:16 PM, Uma Maheswara Rao G <
> hadoop.uma@gmail.com
>     > > wrote:
>     >
>     >> Update: We worked on the review comments and additional JIRAs above
>     >> mentioned.
>     >>
>     >> >1. After the feedbacks from Andrew, Eddy, Xiao in JIRA reviews, we
>     >> planned to take up the support for recursive API support.
> HDFS-12291<
>     >> https://issues.apache.org/jira/browse/HDFS-12291>
>     >>
>     >> We provided the recursive API support now.
>     >>
>     >> >2. Xattr optimizations HDFS-12225<https://issues.apac
>     >> he.org/jira/browse/HDFS-12225>
>     >> Improved this portion as well
>     >>
>     >> >3. Few other review comments already fixed and committed
> HDFS-12214<
>     >> https://issues.apache.org/jira/browse/HDFS-12214>
>     >> Fixed the comments.
>     >>
>     >> We are continuing to test the feature and working so far well. Also
> we
>     >> uploaded a combined patch and got the good QA report.
>     >>
>     >> If there are no further objections, we would like to go for merge
> vote
>     >> tomorrow. Please by default this feature will be disabled.
>     >>
>     >> Regards,
>     >> Uma
>     >>
>     >> On Fri, Aug 18, 2017 at 11:27 PM, Gangumalla, Uma <
>     >> uma.gangumalla@intel.com> wrote:
>     >>
>     >>> Hi Andrew,
>     >>>
>     >>> >Great to hear. It'd be nice to define which use cases are met by
> the
>     >>> current version of SPS, and which will be handled after the merge.
>     >>> After the discussions in JIRA, we planned to support recursive API
> as
>     >>> well. The primary use cases we planned was for Hbase. Please check
> next
>     >>> point for use case details.
>     >>>
>     >>> >A bit more detail in the design doc on how HBase would use this
> feature
>     >>> would also be helpful. Is there an HBase JIRA already?
>     >>> Please find the usecase details at this comment in JIRA:
>     >>> https://issues.apache.org/jira/browse/HDFS-10285?focusedComm
>     >>> entId=16120227&page=com.atlassian.jira.plugin.system.issueta
>     >>> bpanels:comment-tabpanel#comment-16120227
>     >>>
>     >>> >I also spent some more time with the design doc and posted a few
>     >>> questions on the JIRA.
>     >>> Thank you for the reviews.
>     >>>
>     >>> To summarize the discussions in JIRA:
>     >>> 1. After the feedbacks from Andrew, Eddy, Xiao in JIRA reviews, we
>     >>> planned to take up the support for recursive API support.
> HDFS-12291<
>     >>> https://issues.apache.org/jira/browse/HDFS-12291> (Rakesh started
> the
>     >>> work on it)
>     >>> 2. Xattr optimizations HDFS-12225<https://issues.apac
>     >>> he.org/jira/browse/HDFS-12225> (Patch available)
>     >>> 3. Few other review comments already fixed and committed
> HDFS-12214<
>     >>> https://issues.apache.org/jira/browse/HDFS-12214>
>     >>>
>     >>> For tracking the follow-up tasks we filed JIRA HDFS-12226, they
> should
>     >>> not be critical for merge.
>     >>>
>     >>> Regards,
>     >>> Uma
>     >>>
>     >>> From: Andrew Wang <andrew.wang@cloudera.com<mailto:
>     >>> andrew.wang@cloudera.com>>
>     >>> Date: Friday, July 28, 2017 at 11:33 AM
>     >>> To: Uma Gangumalla <uma.gangumalla@intel.com<mailto:
>     >>> uma.gangumalla@intel.com>>
>     >>> Cc: "hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>"
> <
>     >>> hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>>
>     >>> Subject: Re: [DISCUSS] Merge Storage Policy Satisfier (SPS)
> [HDFS-10285]
>     >>> feature branch to trunk
>     >>>
>     >>> Hi Uma,
>     >>>
>     >>> > If there are still plans to make changes that affect
> compatibility
>     >>> (the hybrid RPC and bulk DN work mentioned sound like they would),
> then we
>     >>> can cut branch-3 first, or wait to merge until after these tasks
> are
>     >>> finished.
>     >>> [Uma] We don’t see that 2 items as high priority for the feature.
> Users
>     >>> would be able to use the feature with current code base and API.
> So, we
>     >>> would consider them after branch-3 only. That should be perfectly
> fine IMO.
>     >>> The current API is very much useful for Hbase scenario. In Hbase
> case, they
>     >>> will rename files under to different policy directory. They will
> not set
>     >>> the policies always. So, when rename files under to different
> policy
>     >>> directory, they can simply call satisfyStoragePolicy, they don’t
> need any
>     >>> hybrid API.
>     >>>
>     >>> Great to hear. It'd be nice to define which usecases are met by the
>     >>> current version of SPS, and which will be handled after the merge.
>     >>>
>     >>> A bit more detail in the design doc on how HBase would use this
> feature
>     >>> would also be helpful. Is there an HBase JIRA already?
>     >>>
>     >>> I also spent some more time with the design doc and posted a few
>     >>> questions on the JIRA.
>     >>>
>     >>> Best,
>     >>> Andrew
>     >>>
>     >>
>     >>
>     >
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message