Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: local policy includes SPF record at
 spf.trusted-forwarder.org)
MIME-Version: 1.0
In-Reply-To: 
 <CAJs-t7N9WbPaw+yfdGKfQSjtFeUW2jmXQD9YktQomayLeh0BCg@mail.gmail.com>
References: <A31E1430-33BE-437C-A61E-050F9A67C109@hortonworks.com>
	<CA+4kjVvY_3kr1_jewwPeDZL-Ami5sjVp5Mt72pHemi+7h4cM6A@mail.gmail.com>
	<CABCYYb_Q8c4YHhVpNz-F4MUgpMA1sokkyR9F4Xu3SA-4KCk57A@mail.gmail.com>
	<CAJs-t7N9WbPaw+yfdGKfQSjtFeUW2jmXQD9YktQomayLeh0BCg@mail.gmail.com>
Date: Mon, 11 Nov 2013 10:06:29 -0800
Message-ID: 
 <CABkyDzYgL-DPVcdSX_Ux=sLCWTfNRaROdksQ-W2B7eZR9Sj5HA@mail.gmail.com>
Subject: Re: Next releases
From: Hari Mankude <hmankude@talena-inc.com>
To: mapreduce-dev@hadoop.apache.org
Cc: "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>,
	"common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>,
	"hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=f46d0444048e5f26d004eaea9734

--f46d0444048e5f26d004eaea9734
Content-Type: text/plain; charset=ISO-8859-1

Hi Arun,

Another feature that would be relevant and got deferred was the symlink
work (HADOOP-10020) that Colin and Andrew were working on. Can we include
this in hadoop-2.3.0 also?

thanks
hari


On Sun, Nov 10, 2013 at 2:07 PM, Alejandro Abdelnur <tucu@cloudera.com>wrote:

> Arun, thanks for jumping on this.
>
> On hadoop branch-2.2. I've quickly scanned the commit logs starting from
> the 2.2.0 release and I've found around 20 JIRAs that I like seeing in
> 2.2.1. Not all of them are bugs but the don't shake anything and improve
> usability.
>
> I presume others will have their own laundry lists as well and I wonder the
> union of all of them how much adds up to the current 81 commits.
>
> How about splitting the JIRAs among a few contributors to assert there is
> nothing risky in there? And if so get discuss getting rid of those commits
> for 2.2.1. IMO doing that would be cheaper than selectively applying
> commits on a fresh branch.
>
> Said this, I think we should get 2.2.1 out of the door before switching
> main efforts to 2.3.0. I volunteer myself to drive 2.2.1 a  release if ASAP
> if you don't have the bandwidth at the moment for it.
>
> Cheers.
>
> Alejandro
>
>
> ************************************************************************************
> Commits in branch-2.2 that I'd like them to be in the 2.2.1 release:
>
> The ones prefixed with '*' technically are not bugs.
>
>  YARN-1284. LCE: Race condition leaves dangling cgroups entries for killed
> containers. (Alejandro Abdelnur via Sandy Ryza)
>  YARN-1265. Fair Scheduler chokes on unhealthy node reconnect (Sandy Ryza)
>  YARN-1044. used/min/max resources do not display info in the scheduler
> page (Sangjin Lee via Sandy Ryza)
>  YARN-305. Fair scheduler logs too many "Node offered to app" messages.
> (Lohit Vijayarenu via Sandy Ryza)
> *MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters. (Tzuyoshi Ozawa via Sandy
> Ryza)
>  YARN-1259. In Fair Scheduler web UI, queue num pending and num active apps
> switched. (Robert Kanter via Sandy Ryza)
>  YARN-1295. In UnixLocalWrapperScriptBuilder, using bash -c can cause Text
> file busy errors. (Sandy Ryza)
> *MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write
> out text files without separators (Sandy Ryza)
> *YARN-1258. Allow configuring the Fair Scheduler root queue (Sandy Ryza)
> *YARN-1288. Make Fair Scheduler ACLs more user friendly (Sandy Ryza)
>  YARN-1330. Fair Scheduler: defaultQueueSchedulingPolicy does not take
> effect (Sandy Ryza)
>  HDFS-5403. WebHdfs client cannot communicate with older WebHdfs servers
> post HDFS-5306. Contributed by Aaron T. Myers.
> *YARN-1335. Move duplicate code from FSSchedulerApp and FiCaSchedulerApp
> into SchedulerApplication (Sandy Ryza)
> *YARN-1333. Support blacklisting in the Fair Scheduler (Tsuyoshi Ozawa via
> Sandy Ryza)
> *MAPREDUCE-4680. Job history cleaner should only check timestamps of files
> in old enough directories (Robert Kanter via Sandy Ryza)
>  YARN-1109. Demote NodeManager "Sending out status for container" logs to
> debug (haosdent via Sandy Ryza)
> *YARN-1321. Changed NMTokenCache to support both singleton and an instance
> usage. Contributed by Alejandro Abdelnur
>  YARN-1343. NodeManagers additions/restarts are not reported as node
> updates in AllocateResponse responses to AMs. (tucu)
>  YARN-1381. Same relaxLocality appears twice in exception message of
> AMRMClientImpl#checkLocalityRelaxationConflict() (Ted Yu via Sandy Ryza)
>  HADOOP-9898. Set SO_KEEPALIVE on all our sockets. Contributed by Todd
> Lipcon.
>  YARN-1388. Fair Scheduler page always displays blank fair share (Liyin
> Liang via Sandy Ryza)
>
>
>
> On Fri, Nov 8, 2013 at 10:35 PM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > Arun, what are your thoughts on test-only patches?  I know I've been
> > merging a lot of Windows test stabilization patches down to branch-2.2.
> >  These can't rightly be called blockers, but they do improve dev
> > experience, and there is no risk to product code.
> >
> > Chris Nauroth
> > Hortonworks
> > http://hortonworks.com/
> >
> >
> >
> > On Fri, Nov 8, 2013 at 1:30 AM, Steve Loughran <stevel@hortonworks.com
> > >wrote:
> >
> > > On 8 November 2013 02:42, Arun C Murthy <acm@hortonworks.com> wrote:
> > >
> > > > Gang,
> > > >
> > > >  Thinking through the next couple of releases here, appreciate f/b.
> > > >
> > > >  # hadoop-2.2.1
> > > >
> > > >  I was looking through commit logs and there is a *lot* of content
> here
> > > > (81 commits as on 11/7). Some are features/improvements and some are
> > > fixes
> > > > - it's really hard to distinguish what is important and what isn't.
> > > >
> > > >  I propose we start with a blank slate (i.e. blow away branch-2.2 and
> > > > start fresh from a copy of branch-2.2.0)  and then be very careful
> and
> > > > meticulous about including only *blocker* fixes in branch-2.2. So,
> most
> > > of
> > > > the content here comes via the next minor release (i.e. hadoop-2.3)
> > > >
> > > >  In future, we continue to be *very* parsimonious about what gets
> into
> > a
> > > > patch release (major.minor.patch) - in general, these should be only
> > > > *blocker* fixes or key operational issues.
> > > >
> > >
> > > +1
> > >
> > >
> > > >
> > > >  # hadoop-2.3
> > > >
> > > >  I'd like to propose the following features for YARN/MR to make it
> into
> > > > hadoop-2.3 and punt the rest to hadoop-2.4 and beyond:
> > > >  * Application History Server - This is happening in  a branch and is
> > > > close; with it we can provide a reasonable experience for new
> > frameworks
> > > > being built on top of YARN.
> > > >  * Bug-fixes in RM Restart
> > > >  * Minimal support for long-running applications (e.g. security) via
> > > > YARN-896
> > > >
> > >
> > > +1 -the complete set isn't going to make it, but I'm sure we can
> identify
> > > the key ones
> > >
> > >
> > >
> > > >  * RM Fail-over via ZKFC
> > > >  * Anything else?
> > > >
> > > >  HDFS???
> > > >
> > > >
> > >
> > >    - If I had the time, I'd like to do some work on the HADOOP-9361
> > >    filesystem spec & tests -this is mostly some specification, the
> basis
> > > of a
> > >    better test framework for newer FS tests, and some more tests, with
> a
> > >    couple of minor changes to some of the FS code, mainly in terms of
> > >    tightening some of the exceptions thrown (IOE -> EOF)
> > >
> > > otherwise:
> > >
> > >    - I'd like the hadoop-openstack  JAR in; it's already in branch-2 so
> > >    it's a matter of ensuring testing during the release against as many
> > >    providers as possible.
> > >    - There are a fair few JIRAs about updating versions of dependencies
> > >    -the S3 JetS3t update went in this week, but there are more, as well
> > as
> > >    cruft in the POMs which shows up downstream. I think we could update
> > the
> > >    low-risk dependencies (test-time, log4j, &c), while avoiding those
> we
> > > know
> > >    will be trouble (jetty). This may seem minor but it does make a big
> > > diff to
> > >    the downstream projects.
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
>
>
> --
> Alejandro
>

--f46d0444048e5f26d004eaea9734--