hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gangumalla, Uma" <uma.ganguma...@intel.com>
Subject Re: Hadoop 3.1.0 release discussion
Date Tue, 30 Jan 2018 08:51:49 GMT
Hi Wangda,

Sorry that we have not started vote on 29th. Daryn is reviewing the branch and he needs a
day for his finalized review, then we have to prioritize the comments and decide.
Will keep updated here.

Regards,
Uma


On 1/28/18, 5:31 PM, "Wangda Tan" <wheeleast@gmail.com> wrote:

    Hi Uma,
    
    Thanks, I saw HDFS-13050 has been resolved 4 hours ago, I don't see any
    other blockers under HDFS-10285. I think you guys should be able to start
    voting thread in time for merging to trunk.
    
    - Wangda
    
    On Mon, Jan 29, 2018 at 3:12 AM, Gangumalla, Uma <uma.gangumalla@intel.com>
    wrote:
    
    > Hi Wangda,
    >
    >
    >
    > Sorry for the delay.
    >
    >
    >
    > >>* (Uma) HDFS-10285: HDFS SPS. There're two remaining blockers:
    > HDFS-12995/HDFS-13050. @Uma could you update what's the ETA of the two
    > JIRAs?
    >
    > We have only one blocker now that is HDFS-13050 and we finished the key
    > implementation from HDFS-12995, by HDFS-13075.
    >
    > We are planning to start vote by tomorrow (29th PST time). So, we request
    > you to give us time for running vote. We will keep SPS off by default. So,
    > interested users only can enable explicitly.
    >
    >
    >
    > Regards,
    >
    > Uma
    >
    >
    >
    > *From: *Wangda Tan <wheeleast@gmail.com>
    > *Date: *Friday, January 26, 2018 at 5:21 PM
    > *To: *Uma Gangumalla <uma.gangumalla@intel.com>
    > *Cc: *"hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>, "
    > yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>, "
    > common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>, "
    > mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>, Vinod
    > Kumar Vavilapalli <vinodkv@hortonworks.com>
    > *Subject: *Re: Hadoop 3.1.0 release discussion
    >
    >
    >
    > Hi All,
    >
    >
    >
    > Just a reminder about feature freeze date.
    >
    >
    >
    > Feature freeze date for 3.1.0 release is Jan 30 PST (about 4 days from
    > today), If you've any features which live in a branch and targeted to
    > 3.1.0, please reply this email thread. Ideally, we should finish branch
    > merging before feature freeze date.
    >
    >
    >
    > Here's an updated 3.1.0 feature status:
    >
    >
    >
    > 1. Merged features:
    >
    > * (Sunil) YARN-5881: Support absolute value in CapacityScheduler.
    >
    > * (Wangda) YARN-6223: GPU support on YARN. Features in trunk and works
    > end-to-end.
    >
    > * (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN native services.
    >
    > * (Steve Loughran): HADOOP-13786: S3Guard committer for zero-rename
    > commits.
    >
    > * (Suma): YARN-7117: Capacity Scheduler: Support Auto Creation of Leaf
    > Queues While Doing Queue Mapping.
    >
    > * (Chris Douglas) HDFS-9806: HDFS Tiered Storage.
    >
    > * (Zhankun) YARN-5983: FPGA support. Majority implementations completed
    > and merged to trunk. Except for UI/documentation.
    >
    >
    >
    > 2. Features close to finish:
    >
    > * (Uma) HDFS-10285: HDFS SPS. There're two remaining
    > blockers: HDFS-12995/HDFS-13050. @Uma could you update what's the ETA of
    > the two JIRAs?
    >
    > * (Arun Suresh / Kostas / Wangda). YARN-6592: New SchedulingRequest and
    > anti-affinity support. (Voting thread started).
    >
    >
    >
    > 3. Tentative features:
    >
    > * (Arun Suresh). YARN-5972: Support pausing/freezing opportunistic
    > containers. Only one pending patch. Plan to finish before Jan 7th.
    >
    > * (Haibo Chen). YARN-1011: Resource overcommitment. Looks challenging to
    > be done before Jan 2018.
    >
    > * (Anu): HDFS-7240: Ozone. Given the discussion on HDFS-7240. Looks
    > challenging to be done before Jan 2018.
    >
    > * (Varun V) YARN-5673: container-executor write. Given security
    > refactoring of c-e (YARN-6623) is already landed, IMHO other stuff may be
    > moved to 3.2.
    >
    >
    >
    > Thanks,
    >
    > Wangda
    >
    >
    >
    >
    >
    > On Mon, Jan 22, 2018 at 1:49 PM, Gangumalla, Uma <uma.gangumalla@intel.com>
    > wrote:
    >
    > Sure, Wangda.
    >
    > Regards,
    > Uma
    >
    >
    > On 1/18/18, 10:19 AM, "Wangda Tan" <wheeleast@gmail.com> wrote:
    >
    >     Thanks Uma,
    >
    >     Could you update this thread once the merge vote started?
    >
    >     Best,
    >     Wangda
    >
    >     On Wed, Jan 17, 2018 at 4:30 PM, Gangumalla, Uma <
    > uma.gangumalla@intel.com>
    >     wrote:
    >
    >     > HI Wangda,
    >     >
    >     >  Thank you for the head-up mail.
    >     >  We are in the branch (HDFS-10285) and trying to push the tasks
    > sooner
    >     > before the deadline.
    >     >
    >     > Regards,
    >     > Uma
    >     >
    >     > On 1/17/18, 11:35 AM, "Wangda Tan" <wheeleast@gmail.com> wrote:
    >     >
    >     >     Hi All,
    >     >
    >     >     Since we're fast approaching previously proposed feature freeze
    > date
    >     > (Jan
    >     >     30, about 13 days from today). If you've any features which live
    > in a
    >     >     branch and targeted to 3.1.0, please reply this email thread.
    > Ideally,
    >     > we
    >     >     should finish branch merging before feature freeze date.
    >     >
    >     >     Here's an updated 3.1.0 feature status:
    >     >
    >     >     1. Merged & Completed features:
    >     >     * (Sunil) YARN-5881: Support absolute value in CapacityScheduler.
    >     >     * (Wangda) YARN-6223: GPU support on YARN. Features in trunk and
    > works
    >     >     end-to-end.
    >     >     * (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN native
    > services.
    >     >     * (Steve Loughran): HADOOP-13786: S3Guard committer for
    > zero-rename
    >     > commits.
    >     >     * (Suma): YARN-7117: Capacity Scheduler: Support Auto Creation
    > of Leaf
    >     >     Queues While Doing Queue Mapping.
    >     >     * (Chris Douglas) HDFS-9806: HDFS Tiered Storage.
    >     >
    >     >     2. Features close to finish:
    >     >     * (Zhankun) YARN-5983: FPGA support. Majority implementations
    >     > completed and
    >     >     merged to trunk. Except for UI/documentation.
    >     >     * (Uma) HDFS-10285: HDFS SPS. Majority implementations are done,
    > some
    >     >     discussions going on about implementation.
    >     >     * (Arun Suresh / Kostas / Wangda). YARN-6592: New
    > SchedulingRequest and
    >     >     anti-affinity support. Close to finish, on track to be merged
    > before
    >     > Jan 30.
    >     >
    >     >     3. Tentative features:
    >     >     * (Arun Suresh). YARN-5972: Support pausing/freezing
    > opportunistic
    >     >     containers. Only one pending patch. Plan to finish before Jan
    > 7th.
    >     >     * (Haibo Chen). YARN-1011: Resource overcommitment. Looks
    > challenging
    >     > to be
    >     >     done before Jan 2018.
    >     >     * (Anu): HDFS-7240: Ozone. Given the discussion on HDFS-7240.
    > Looks
    >     >     challenging to be done before Jan 2018.
    >     >     * (Varun V) YARN-5673: container-executor write. Given security
    >     > refactoring
    >     >     of c-e (YARN-6623) is already landed, IMHO other stuff may be
    > moved to
    >     > 3.2.
    >     >
    >     >     Thanks,
    >     >     Wangda
    >     >
    >     >
    >     >
    >     >
    >     >     On Fri, Dec 15, 2017 at 1:20 PM, Wangda Tan <wheeleast@gmail.com
    > >
    >     > wrote:
    >     >
    >     >     > Hi all,
    >     >     >
    >     >     > Congratulations on the 3.0.0-GA release!
    >     >     >
    >     >     > As we discussed in the previous email thread [1], I'd like to
    > restart
    >     >     > 3.1.0 release plans.
    >     >     >
    >     >     > a) Quick summary:
    >     >     > a.1 Release status
    >     >     > We started 3.1 release discussion on Sep 6, 2017 [1]. As of
    > today,
    >     >     > there’re 232 patches loaded on 3.1.0 alone [2], besides 6 open
    >     > blockers and
    >     >     > 22 open critical issues.
    >     >     >
    >     >     > a.2 Release date update
    >     >     > Considering delays of 3.0-GA release by month-and-a-half, I
    > propose
    >     > to
    >     >     > move the dates as follows
    >     >     >  - feature freeze date from Dec 15, 2017, to Jan 30, 2018 -
    > last
    >     > date for
    >     >     > any branches to get merged too;
    >     >     >  - code freeze (blockers & critical only) date to Feb 08, 2018;
    >     >     >  - release voting start by Feb 18, 2018, leaving time for at
    > least
    >     > two RCx
    >     >     >  - release date from Jan 15, 2018, to Feb 28, 2018;
    >     >     >
    >     >     > Unlike before, I added an additional milestone for
    >     > release-vote-start so
    >     >     > that we can account for voting time-period also.
    >     >     >
    >     >     > This overall is still 5 1/2 months of release-timeline unlike
    > the
    >     > faster
    >     >     > cadence we hoped for, but this, in my opinion, is the
    > best-updated
    >     > timeline
    >     >     > given the delays of the final release of 3.0-GA.
    >     >     >
    >     >     > b) Individual feature status:
    >     >     > I spoke to several feature owners and checked the status of
    >     > un-finished
    >     >     > features, following are status of features planned to 3.1.0:
    >     >     >
    >     >     > b.1 Merged & Completed features:
    >     >     > * (Sunil) YARN-5881: Support absolute value in
    > CapacityScheduler.
    >     >     > * (Wangda) YARN-6223: GPU support on YARN. Features in trunk
    > and
    >     > works
    >     >     > end-to-end.
    >     >     > * (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN native
    >     > services.
    >     >     > * (Steve Loughran): HADOOP-13786: S3Guard committer for
    > zero-rename
    >     >     > commits.
    >     >     > * (Suma): YARN-7117: Capacity Scheduler: Support Auto Creation
    > of
    >     > Leaf
    >     >     > Queues While Doing Queue Mapping.
    >     >     >
    >     >     > b.2 Features close to finish:
    >     >     > * (Chris Douglas) HDFS-9806: HDFS Tiered Storage. Being voting
    > now.
    >     >     > * (Zhankun) YARN-5983: FPGA support. Majority implementations
    >     > completed
    >     >     > and merged to trunk. Except for UI/documentation.
    >     >     > * (Uma) HDFS-10285: HDFS SPS. Majority implementations are
    > done, some
    >     >     > discussions going on about implementation.
    >     >     >
    >     >     > b.3 Tentative features:
    >     >     > * (Arun Suresh). YARN-5972: Support pausing/freezing
    > opportunistic
    >     >     > containers. Only one pending patch. Plan to finish before Jan
    > 7th.
    >     >     > * (Haibo Chen). YARN-1011: Resource overcommitment. Looks
    >     > challenging to
    >     >     > be done before Jan 2018.
    >     >     > * (Arun Suresh / Kostas / Wangda). YARN-6592: New
    > SchedulingRequest
    >     > and
    >     >     > anti-affinity support. Tentative will figure out by Jan 1st.
    >     >     > * (Anu): HDFS-7240: Ozone. Given the discussion on HDFS-7240.
    > Looks
    >     >     > challenging to be done before Jan 2018.
    >     >     > * (Varun V) YARN-5673: container-executor write. Given security
    >     >     > refactoring of c-e (YARN-6623) is already landed, IMHO other
    > stuff
    >     > may be
    >     >     > moved to 3.2.
    >     >     >
    >     >     > b.4 Additional release drivers
    >     >     > * More exhaustive upgrade testing from 2.x to 3.x.
    >     >     >
    >     >     > c) Regarding branch cut:
    >     >     >
    >     >     > We will keep pointing trunk to 3.1 and cut branch-3.1 until:
    > A. some
    >     >     > feature planned to 3.2 has to be landed on trunk or B. After
    > feature
    >     > freeze
    >     >     > date, whichever comes first.
    >     >     >
    >     >     > I've also talked offline with Vinod to get help on
    > release-management
    >     >     > given this is my first release. He agreed to help do this
    > release
    >     > jointly.
    >     >     >
    >     >     > Thoughts?
    >     >     >
    >     >     > Thanks,
    >     >     > Wangda Tan
    >     >     >
    >     >     > [1] https://lists.apache.org/thread.html/
    >     > c11506c3250c9481852130616b3cb0
    >     >     > 9a0e222f5c2465c015f9906dab@%3Cyarn-dev.hadoop.apache.org%3E
    >     >     > [2] "project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion
    > in
    >     > (3.1.0)
    >     >     > AND fixVersion not in (3.0.0,2.9.0) ORDER BY priority DESC”
    >     >     >
    >     >     >
    >     >
    >     >
    >     >
    >
    >
    >
    


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Mime
View raw message