hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Kanter <rkan...@cloudera.com.INVALID>
Subject Re: Hadoop 3.2 Release Plan proposal
Date Tue, 02 Oct 2018 23:52:01 GMT
Looks like there's two that weren't updated:
>> [115] 16:32 : hadoop-common (trunk) :: grep "3.2.0-SNAPSHOT" . -r
--include=pom.xml
./hadoop-project/pom.xml:
<hadoop.assemblies.version>3.2.0-SNAPSHOT</hadoop.assemblies.version>
./pom.xml:    <hadoop.version>3.2.0-SNAPSHOT</hadoop.version>

I've just pushed in an addendum commit to fix those.
In the future, please make sure to do a sanity compile when updating poms.

thanks
- Robert

On Tue, Oct 2, 2018 at 11:44 AM Aaron Fabbri <fabbri@cloudera.com.invalid>
wrote:

> Trunk is not building for me.. Did you miss a 3.2.0-SNAPSHOT in the
> top-level pom.xml?
>
>
> On Tue, Oct 2, 2018 at 10:16 AM Sunil G <sunilg@apache.org> wrote:
>
> > Hi All
> >
> > As mentioned in earlier mail, I have cut branch-3.2 and reset trunk to
> > 3.3.0-SNAPSHOT. I will share the RC details sooner once all necessary
> > patches are pulled into branch-3.2.
> >
> > Thank You
> > - Sunil
> >
> >
> > On Mon, Sep 24, 2018 at 2:00 PM Sunil G <sunilg@apache.org> wrote:
> >
> > > Hi All
> > >
> > > We are now down to the last Blocker and HADOOP-15407 is merged to
> trunk.
> > > Thanks for the support.
> > >
> > > *Plan for RC*
> > > 3.2 branch cut and reset trunk : *25th Tuesday*
> > > RC0 for 3.2: *28th Friday*
> > >
> > > Thank You
> > > Sunil
> > >
> > >
> > > On Mon, Sep 17, 2018 at 3:21 PM Sunil G <sunilg@apache.org> wrote:
> > >
> > >> Hi All
> > >>
> > >> We are down to 3 Blockers and 4 Critical now. Thanks all of you for
> > >> helping in this. I am following up on these tickets, once its closed
> we
> > >> will cut the 3.2 branch.
> > >>
> > >> Thanks
> > >> Sunil Govindan
> > >>
> > >>
> > >> On Wed, Sep 12, 2018 at 5:10 PM Sunil G <sunilg@apache.org> wrote:
> > >>
> > >>> Hi All,
> > >>>
> > >>> Inline with the original 3.2 communication proposal dated 17th July
> > >>> 2018, I would like to provide more updates.
> > >>>
> > >>> We are approaching previously proposed code freeze date (September
> 14,
> > >>> 2018). So I would like to cut 3.2 branch on 17th Sept and point
> > existing
> > >>> trunk to 3.3 if there are no issues.
> > >>>
> > >>> *Current Release Plan:*
> > >>> Feature freeze date : all features to merge by September 7, 2018.
> > >>> Code freeze date : blockers/critical only, no improvements and
> > >>> blocker/critical bug-fixes September 14, 2018.
> > >>> Release date: September 28, 2018
> > >>>
> > >>> If any critical/blocker tickets which are targeted to 3.2.0, we need
> to
> > >>> backport to 3.2 post branch cut.
> > >>>
> > >>> Here's an updated 3.2.0 feature status:
> > >>>
> > >>> 1. Merged & Completed features:
> > >>>
> > >>> - (Wangda) YARN-8561: Hadoop Submarine project for DeepLearning
> > >>> workloads Initial cut.
> > >>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
> > >>> - (Sunil) YARN-7494: Multi Node scheduling support in Capacity
> > >>> Scheduler.
> > >>> - (Chandni/Eric) YARN-7512: Support service upgrade via YARN Service
> > API
> > >>> and CLI.
> > >>> - (Naga/Sunil) YARN-3409: Node Attributes support in YARN.
> > >>> - (Inigo) HDFS-12615: Router-based HDFS federation. Improvement
> works.
> > >>>
> > >>> 2. Features close to finish:
> > >>>
> > >>> - (Steve) S3Guard Phase III. Close to commit.
> > >>> - (Steve) S3a phase V. Close to commit.
> > >>> - (Steve) Support Windows Azure Storage. Close to commit.
> > >>>
> > >>> 3. Tentative/Cancelled features for 3.2:
> > >>> - (Rohith) YARN-5742: Serve aggregated logs of historical apps from
> > >>> ATSv2. Patch in progress.
> > >>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks challenging
> to
> > >>> be done before Aug 2018.
> > >>> - (Eric) YARN-7129: Application Catalog for YARN applications.
> > >>> Challenging as more discussions are on-going.
> > >>>
> > >>> *Summary of 3.2.0 issues status:*
> > >>> 19 Blocker and Critical issues [1] are open, I am following up with
> > >>> owners to get status on each of them to get in by Code Freeze date.
> > >>>
> > >>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in
> > (Blocker,
> > >>> Critical) AND resolution = Unresolved AND "Target Version/s" = 3.2.0
> > ORDER
> > >>> BY priority DESC
> > >>>
> > >>> Thanks,
> > >>> Sunil
> > >>>
> > >>>
> > >>>
> > >>> On Thu, Aug 30, 2018 at 9:59 PM Sunil G <sunilg@apache.org> wrote:
> > >>>
> > >>>> Hi All,
> > >>>>
> > >>>> Inline with earlier communication dated 17th July 2018, I would
like
> > to
> > >>>> provide some updates.
> > >>>>
> > >>>> We are approaching previously proposed code freeze date (Aug 31).
> > >>>>
> > >>>> One of the critical feature Node Attributes feature merge
> > >>>> discussion/vote is ongoing. Also few other Blocker bugs need a
bit
> > more
> > >>>> time. With regard to this, suggesting to push the feature/code
> freeze
> > for 2
> > >>>> more weeks to accommodate these jiras too.
> > >>>>
> > >>>> Proposing Updated changes in plan inline with this:
> > >>>> Feature freeze date : all features to merge by September 7, 2018.
> > >>>> Code freeze date : blockers/critical only, no improvements and
> > >>>>  blocker/critical bug-fixes September 14, 2018.
> > >>>> Release date: September 28, 2018
> > >>>>
> > >>>> If any features in branch which are targeted to 3.2.0, please reply
> to
> > >>>> this email thread.
> > >>>>
> > >>>> *Here's an updated 3.2.0 feature status:*
> > >>>>
> > >>>> 1. Merged & Completed features:
> > >>>>
> > >>>> - (Wangda) YARN-8561: Hadoop Submarine project for DeepLearning
> > >>>> workloads Initial cut.
> > >>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
> > >>>> - (Sunil) YARN-7494: Multi Node scheduling support in Capacity
> > >>>> Scheduler.
> > >>>> - (Chandni/Eric) YARN-7512: Support service upgrade via YARN Service
> > >>>> API and CLI.
> > >>>>
> > >>>> 2. Features close to finish:
> > >>>>
> > >>>> - (Naga/Sunil) YARN-3409: Node Attributes support in YARN.
> Merge/Vote
> > >>>> Ongoing.
> > >>>> - (Rohith) YARN-5742: Serve aggregated logs of historical apps
from
> > >>>> ATSv2. Patch in progress.
> > >>>> - (Virajit) HDFS-12615: Router-based HDFS federation. Improvement
> > works.
> > >>>> - (Steve) S3Guard Phase III, S3a phase V, Support Windows Azure
> > >>>> Storage. In progress.
> > >>>>
> > >>>> 3. Tentative features:
> > >>>>
> > >>>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks challenging
> > to
> > >>>> be done before Aug 2018.
> > >>>> - (Eric) YARN-7129: Application Catalog for YARN applications.
> > >>>> Challenging as more discussions are on-going.
> > >>>>
> > >>>> *Summary of 3.2.0 issues status:*
> > >>>>
> > >>>> 26 Blocker and Critical issues [1] are open, I am following up
with
> > >>>> owners to get status on each of them to get in by Code Freeze date.
> > >>>>
> > >>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in
> > >>>> (Blocker, Critical) AND resolution = Unresolved AND "Target
> > Version/s" =
> > >>>> 3.2.0 ORDER BY priority DESC
> > >>>>
> > >>>> Thanks,
> > >>>> Sunil
> > >>>>
> > >>>> On Tue, Aug 14, 2018 at 10:30 PM Sunil G <sunilg@apache.org>
wrote:
> > >>>>
> > >>>>> Hi All,
> > >>>>>
> > >>>>> Thanks for the feedbacks. Inline with earlier communication
dated
> > 17th
> > >>>>> July 2018, I would like to provide some updates.
> > >>>>>
> > >>>>> We are approaching previously proposed feature freeze date
(Aug 21,
> > >>>>> about 7 days from today).
> > >>>>> If any features in branch which are targeted to 3.2.0, please
reply
> > to
> > >>>>> this email thread.
> > >>>>> Steve has mentioned about the s3 features which will come close
to
> > >>>>> Code Freeze Date (Aug 31st).
> > >>>>>
> > >>>>> *Here's an updated 3.2.0 feature status:*
> > >>>>>
> > >>>>> 1. Merged & Completed features:
> > >>>>>
> > >>>>> - (Wangda) YARN-8561: Hadoop Submarine project for DeepLearning
> > >>>>> workloads Initial cut.
> > >>>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
> > >>>>>
> > >>>>> 2. Features close to finish:
> > >>>>>
> > >>>>> - (Naga/Sunil) YARN-3409: Node Attributes support in YARN.
Major
> > >>>>> patches are all in, only one last
> > >>>>> patch is in review state.
> > >>>>> - (Sunil) YARN-7494: Multi Node scheduling support in Capacity
> > >>>>> Scheduler. Close to commit.
> > >>>>> - (Chandni/Eric) YARN-7512: Support service upgrade via YARN
> Service
> > >>>>> API and CLI. 2 patches are pending
> > >>>>> which will be closed by Feature freeze date.
> > >>>>> - (Rohith) YARN-5742: Serve aggregated logs of historical apps
from
> > >>>>> ATSv2. Patch in progress.
> > >>>>> - (Virajit) HDFS-12615: Router-based HDFS federation. Improvement
> > >>>>> works.
> > >>>>> - (Steve) S3Guard Phase III, S3a phase V, Support Windows Azure
> > >>>>> Storage. In progress.
> > >>>>>
> > >>>>> 3. Tentative features:
> > >>>>>
> > >>>>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
> challenging
> > >>>>> to be done before Aug 2018.
> > >>>>> - (Eric) YARN-7129: Application Catalog for YARN applications.
> > >>>>> Challenging as more discussions are on-going.
> > >>>>>
> > >>>>> *Summary of 3.2.0 issues status:*
> > >>>>>
> > >>>>> 39 Blocker and Critical issues [1] are open, I am checking
with
> > owners
> > >>>>> to get status on each of them to get in by Code Freeze date.
> > >>>>>
> > >>>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority
in
> > >>>>> (Blocker, Critical) AND resolution = Unresolved AND "Target
> > Version/s" =
> > >>>>> 3.2.0 ORDER BY priority DESC
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Sunil
> > >>>>>
> > >>>>> On Fri, Jul 20, 2018 at 8:03 AM Sunil G <sunilg@apache.org>
wrote:
> > >>>>>
> > >>>>>> Thanks Subru for the thoughts.
> > >>>>>> One of the main reason for a major release is to push out
critical
> > >>>>>> features with a faster cadence to the users. If we are
pulling
> more
> > and
> > >>>>>> more different types of features to a minor release, that
branch
> > will
> > >>>>>> become more destabilized and it may be tough to say that
3.1.2 is
> > stable
> > >>>>>> that 3.1.1 for eg. We always tend to improve and stabilize
> features
> > in
> > >>>>>> subsequent minor release.
> > >>>>>> For few companies, it makes sense to push out these new
features
> > >>>>>> faster to make a reach to the users. Adding to the point
to the
> > backporting
> > >>>>>> issues, I agree that its a pain and we can workaround that
with
> > some git
> > >>>>>> scripts. If we can make such scripts available to committers,
> > backport will
> > >>>>>> be seem-less across branches and we can achieve the faster
release
> > cadence
> > >>>>>> also.
> > >>>>>>
> > >>>>>> Thoughts?
> > >>>>>>
> > >>>>>> - Sunil
> > >>>>>>
> > >>>>>>
> > >>>>>> On Fri, Jul 20, 2018 at 3:37 AM Subru Krishnan <subru@apache.org>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Thanks Sunil for volunteering to lead the release effort.
I am
> > >>>>>>> generally
> > >>>>>>> supportive of a release but -1 on a 3.2 (prefer a 3.1.x)
as feel
> we
> > >>>>>>> already
> > >>>>>>> have too many branches to be maintained. I already
see many
> commits
> > >>>>>>> are in
> > >>>>>>> different branches with no apparent rationale, for
e.g: 3.1 has
> > >>>>>>> commits
> > >>>>>>> which are absent in 3.0 etc.
> > >>>>>>>
> > >>>>>>> Additionally AFAIK 3.x has not been deployed in any
major
> > production
> > >>>>>>> setting so the cost of adding features should be minimal.
> > >>>>>>>
> > >>>>>>> Thoughts?
> > >>>>>>>
> > >>>>>>> -Subru
> > >>>>>>>
> > >>>>>>> On Thu, Jul 19, 2018 at 12:31 AM, Sunil G <sunilg@apache.org>
> > wrote:
> > >>>>>>>
> > >>>>>>> > Thanks Steve, Aaron, Wangda for sharing thoughts.
> > >>>>>>> >
> > >>>>>>> > Yes, important changes and features are much needed,
hence we
> > will
> > >>>>>>> be
> > >>>>>>> > keeping the door open for them as possible. Also
considering
> few
> > >>>>>>> more
> > >>>>>>> > offline requests from other folks, I think extending
the
> > timeframe
> > >>>>>>> by
> > >>>>>>> > couple of weeks makes sense (including a second
RC buffer) and
> > >>>>>>> this should
> > >>>>>>> > ideally help us to ship this by September itself.
> > >>>>>>> >
> > >>>>>>> > Revised dates (I will be updating same in Roadmap
wiki as well)
> > >>>>>>> >
> > >>>>>>> > - Feature freeze date : all features to merge
by August 21,
> 2018.
> > >>>>>>> >
> > >>>>>>> > - Code freeze date : blockers/critical only, no
improvements
> and
> > >>>>>>> non
> > >>>>>>> > blocker/critical
> > >>>>>>> >
> > >>>>>>> > bug-fixes  August 31, 2018.
> > >>>>>>> >
> > >>>>>>> > - Release date: September 15, 2018
> > >>>>>>> >
> > >>>>>>> > Thank Eric and Zian, I think Wangda has already
answered your
> > >>>>>>> questions.
> > >>>>>>> >
> > >>>>>>> > Thanks
> > >>>>>>> > Sunil
> > >>>>>>> >
> > >>>>>>> >
> > >>>>>>> > On Thu, Jul 19, 2018 at 12:13 PM Wangda Tan <
> wheeleast@gmail.com
> > >
> > >>>>>>> wrote:
> > >>>>>>> >
> > >>>>>>> > > Thanks Sunil for volunteering to be RM of
3.2 release, +1 for
> > >>>>>>> that.
> > >>>>>>> > >
> > >>>>>>> > > To concerns from Steve,
> > >>>>>>> > >
> > >>>>>>> > > It is a good idea to keep the door open to
get important
> > changes
> > >>>>>>> /
> > >>>>>>> > > features in before cutoff. I would prefer
to keep the
> proposed
> > >>>>>>> release
> > >>>>>>> > date
> > >>>>>>> > > to make sure things can happen earlier instead
of last minute
> > >>>>>>> and we all
> > >>>>>>> > > know that releases are always get delayed
:). I'm also fine
> if
> > >>>>>>> we want
> > >>>>>>> > get
> > >>>>>>> > > another several weeks time.
> > >>>>>>> > >
> > >>>>>>> > > Regarding of 3.3 release, I would suggest
doing that before
> > >>>>>>> thanksgiving.
> > >>>>>>> > > Do you think is it good or too early / late?
> > >>>>>>> > >
> > >>>>>>> > > Eric,
> > >>>>>>> > >
> > >>>>>>> > > The YARN-8220 will be replaced by YARN-8135,
if YARN-8135 can
> > >>>>>>> get merged
> > >>>>>>> > > in time, we probably not need the YARN-8220.
> > >>>>>>> > >
> > >>>>>>> > > Sunil,
> > >>>>>>> > >
> > >>>>>>> > > Could u update
> > >>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> > >>>>>>> > Roadmap
> > >>>>>>> > > with proposed plan as well? We can fill feature
list first
> > >>>>>>> before getting
> > >>>>>>> > > consensus of time.
> > >>>>>>> > >
> > >>>>>>> > > Thanks,
> > >>>>>>> > > Wangda
> > >>>>>>> > >
> > >>>>>>> > > On Wed, Jul 18, 2018 at 6:20 PM Aaron Fabbri
> > >>>>>>> <fabbri@cloudera.com.invalid
> > >>>>>>> > >
> > >>>>>>> > > wrote:
> > >>>>>>> > >
> > >>>>>>> > >> On Tue, Jul 17, 2018 at 7:21 PM Steve
Loughran <
> > >>>>>>> stevel@hortonworks.com>
> > >>>>>>> > >> wrote:
> > >>>>>>> > >>
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >> > On 16 Jul 2018, at 23:45, Sunil
G <sunilg@apache.org
> > <mailto:
> > >>>>>>> > >> > sunilg@apache.org>> wrote:
> > >>>>>>> > >> >
> > >>>>>>> > >> > I would also would like to take
this opportunity to come
> up
> > >>>>>>> with a
> > >>>>>>> > >> detailed
> > >>>>>>> > >> > plan.
> > >>>>>>> > >> >
> > >>>>>>> > >> > - Feature freeze date : all features
should be merged by
> > >>>>>>> August 10,
> > >>>>>>> > >> 2018.
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >> > <snip>
> > >>>>>>> > >>
> > >>>>>>> > >> >
> > >>>>>>> > >> > Please let me know if I missed any
features targeted to
> 3.2
> > >>>>>>> per this
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >> > Well there these big todo lists
for S3 & S3Guard.
> > >>>>>>> > >> >
> > >>>>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15226
> > >>>>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15220
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >> > There's a bigger bit of work coming
on for Azure Datalake
> > Gen
> > >>>>>>> 2
> > >>>>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15407
> > >>>>>>> > >> >
> > >>>>>>> > >> > I don't think this is quite ready
yet, I've been doing
> work
> > >>>>>>> on it, but
> > >>>>>>> > >> if
> > >>>>>>> > >> > we have a 3 week deadline, I'm going
to expect some timely
> > >>>>>>> reviews on
> > >>>>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15546
> > >>>>>>> > >> >
> > >>>>>>> > >> > I've uprated that to a blocker feature;
will review the
> S3 &
> > >>>>>>> S3Guard
> > >>>>>>> > >> JIRAs
> > >>>>>>> > >> > to see which of those are blocking.
Then there are some
> > >>>>>>> pressing
> > >>>>>>> > "guave,
> > >>>>>>> > >> > java 9 prep"
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >>  I can help with this part if you like.
> > >>>>>>> > >>
> > >>>>>>> > >>
> > >>>>>>> > >>
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >> > timeline. I would like to volunteer
myself as release
> > manager
> > >>>>>>> of 3.2.0
> > >>>>>>> > >> > release.
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >> > well volunteered!
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >> Yes, thank you for stepping up.
> > >>>>>>> > >>
> > >>>>>>> > >>
> > >>>>>>> > >> >
> > >>>>>>> > >> > I think this raises a good q: what
timetable should we
> have
> > >>>>>>> for the
> > >>>>>>> > >> 3.2. &
> > >>>>>>> > >> > 3.3 releases; if we do want a faster
cadence, then having
> > the
> > >>>>>>> outline
> > >>>>>>> > >> time
> > >>>>>>> > >> > from the 3.2 to the 3.3 release
means that there's less
> > >>>>>>> concern about
> > >>>>>>> > >> > things not making the 3.2 dealine
> > >>>>>>> > >> >
> > >>>>>>> > >> > -Steve
> > >>>>>>> > >> >
> > >>>>>>> > >> >
> > >>>>>>> > >> Good idea to mitigate the short deadline.
> > >>>>>>> > >>
> > >>>>>>> > >> -AF
> > >>>>>>> > >>
> > >>>>>>> > >
> > >>>>>>> >
> > >>>>>>>
> > >>>>>>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message