Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of tucu@cloudera.com designates
 209.85.128.44 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CABCYYb_Q8c4YHhVpNz-F4MUgpMA1sokkyR9F4Xu3SA-4KCk57A@mail.gmail.com>
References: <A31E1430-33BE-437C-A61E-050F9A67C109@hortonworks.com>
 <CA+4kjVvY_3kr1_jewwPeDZL-Ami5sjVp5Mt72pHemi+7h4cM6A@mail.gmail.com>
 <CABCYYb_Q8c4YHhVpNz-F4MUgpMA1sokkyR9F4Xu3SA-4KCk57A@mail.gmail.com>
From: Alejandro Abdelnur <tucu@cloudera.com>
Date: Sun, 10 Nov 2013 14:07:52 -0800
Message-ID: 
 <CAJs-t7N9WbPaw+yfdGKfQSjtFeUW2jmXQD9YktQomayLeh0BCg@mail.gmail.com>
Subject: Re: Next releases
To: "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>
Cc: "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>,
	"common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>,
	"hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=001a11c2c2fc979fdf04ead9da53

--001a11c2c2fc979fdf04ead9da53
Content-Type: text/plain; charset=ISO-8859-1

Arun, thanks for jumping on this.

On hadoop branch-2.2. I've quickly scanned the commit logs starting from
the 2.2.0 release and I've found around 20 JIRAs that I like seeing in
2.2.1. Not all of them are bugs but the don't shake anything and improve
usability.

I presume others will have their own laundry lists as well and I wonder the
union of all of them how much adds up to the current 81 commits.

How about splitting the JIRAs among a few contributors to assert there is
nothing risky in there? And if so get discuss getting rid of those commits
for 2.2.1. IMO doing that would be cheaper than selectively applying
commits on a fresh branch.

Said this, I think we should get 2.2.1 out of the door before switching
main efforts to 2.3.0. I volunteer myself to drive 2.2.1 a  release if ASAP
if you don't have the bandwidth at the moment for it.

Cheers.

Alejandro

************************************************************************************
Commits in branch-2.2 that I'd like them to be in the 2.2.1 release:

The ones prefixed with '*' technically are not bugs.

 YARN-1284. LCE: Race condition leaves dangling cgroups entries for killed
containers. (Alejandro Abdelnur via Sandy Ryza)
 YARN-1265. Fair Scheduler chokes on unhealthy node reconnect (Sandy Ryza)
 YARN-1044. used/min/max resources do not display info in the scheduler
page (Sangjin Lee via Sandy Ryza)
 YARN-305. Fair scheduler logs too many "Node offered to app" messages.
(Lohit Vijayarenu via Sandy Ryza)
*MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters. (Tzuyoshi Ozawa via Sandy
Ryza)
 YARN-1259. In Fair Scheduler web UI, queue num pending and num active apps
switched. (Robert Kanter via Sandy Ryza)
 YARN-1295. In UnixLocalWrapperScriptBuilder, using bash -c can cause Text
file busy errors. (Sandy Ryza)
*MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write
out text files without separators (Sandy Ryza)
*YARN-1258. Allow configuring the Fair Scheduler root queue (Sandy Ryza)
*YARN-1288. Make Fair Scheduler ACLs more user friendly (Sandy Ryza)
 YARN-1330. Fair Scheduler: defaultQueueSchedulingPolicy does not take
effect (Sandy Ryza)
 HDFS-5403. WebHdfs client cannot communicate with older WebHdfs servers
post HDFS-5306. Contributed by Aaron T. Myers.
*YARN-1335. Move duplicate code from FSSchedulerApp and FiCaSchedulerApp
into SchedulerApplication (Sandy Ryza)
*YARN-1333. Support blacklisting in the Fair Scheduler (Tsuyoshi Ozawa via
Sandy Ryza)
*MAPREDUCE-4680. Job history cleaner should only check timestamps of files
in old enough directories (Robert Kanter via Sandy Ryza)
 YARN-1109. Demote NodeManager "Sending out status for container" logs to
debug (haosdent via Sandy Ryza)
*YARN-1321. Changed NMTokenCache to support both singleton and an instance
usage. Contributed by Alejandro Abdelnur
 YARN-1343. NodeManagers additions/restarts are not reported as node
updates in AllocateResponse responses to AMs. (tucu)
 YARN-1381. Same relaxLocality appears twice in exception message of
AMRMClientImpl#checkLocalityRelaxationConflict() (Ted Yu via Sandy Ryza)
 HADOOP-9898. Set SO_KEEPALIVE on all our sockets. Contributed by Todd
Lipcon.
 YARN-1388. Fair Scheduler page always displays blank fair share (Liyin
Liang via Sandy Ryza)


On Fri, Nov 8, 2013 at 10:35 PM, Chris Nauroth <cnauroth@hortonworks.com>wrote:

> Arun, what are your thoughts on test-only patches?  I know I've been
> merging a lot of Windows test stabilization patches down to branch-2.2.
>  These can't rightly be called blockers, but they do improve dev
> experience, and there is no risk to product code.
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Fri, Nov 8, 2013 at 1:30 AM, Steve Loughran <stevel@hortonworks.com
> >wrote:
>
> > On 8 November 2013 02:42, Arun C Murthy <acm@hortonworks.com> wrote:
> >
> > > Gang,
> > >
> > >  Thinking through the next couple of releases here, appreciate f/b.
> > >
> > >  # hadoop-2.2.1
> > >
> > >  I was looking through commit logs and there is a *lot* of content here
> > > (81 commits as on 11/7). Some are features/improvements and some are
> > fixes
> > > - it's really hard to distinguish what is important and what isn't.
> > >
> > >  I propose we start with a blank slate (i.e. blow away branch-2.2 and
> > > start fresh from a copy of branch-2.2.0)  and then be very careful and
> > > meticulous about including only *blocker* fixes in branch-2.2. So, most
> > of
> > > the content here comes via the next minor release (i.e. hadoop-2.3)
> > >
> > >  In future, we continue to be *very* parsimonious about what gets into
> a
> > > patch release (major.minor.patch) - in general, these should be only
> > > *blocker* fixes or key operational issues.
> > >
> >
> > +1
> >
> >
> > >
> > >  # hadoop-2.3
> > >
> > >  I'd like to propose the following features for YARN/MR to make it into
> > > hadoop-2.3 and punt the rest to hadoop-2.4 and beyond:
> > >  * Application History Server - This is happening in  a branch and is
> > > close; with it we can provide a reasonable experience for new
> frameworks
> > > being built on top of YARN.
> > >  * Bug-fixes in RM Restart
> > >  * Minimal support for long-running applications (e.g. security) via
> > > YARN-896
> > >
> >
> > +1 -the complete set isn't going to make it, but I'm sure we can identify
> > the key ones
> >
> >
> >
> > >  * RM Fail-over via ZKFC
> > >  * Anything else?
> > >
> > >  HDFS???
> > >
> > >
> >
> >    - If I had the time, I'd like to do some work on the HADOOP-9361
> >    filesystem spec & tests -this is mostly some specification, the basis
> > of a
> >    better test framework for newer FS tests, and some more tests, with a
> >    couple of minor changes to some of the FS code, mainly in terms of
> >    tightening some of the exceptions thrown (IOE -> EOF)
> >
> > otherwise:
> >
> >    - I'd like the hadoop-openstack  JAR in; it's already in branch-2 so
> >    it's a matter of ensuring testing during the release against as many
> >    providers as possible.
> >    - There are a fair few JIRAs about updating versions of dependencies
> >    -the S3 JetS3t update went in this week, but there are more, as well
> as
> >    cruft in the POMs which shows up downstream. I think we could update
> the
> >    low-risk dependencies (test-time, log4j, &c), while avoiding those we
> > know
> >    will be trouble (jetty). This may seem minor but it does make a big
> > diff to
> >    the downstream projects.
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>


-- 
Alejandro

--001a11c2c2fc979fdf04ead9da53--