hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@apache.org>
Subject Re: [DISCUSS] Looking to a 2.8.0 release
Date Wed, 25 Nov 2015 19:30:18 GMT

> There's a lot of stuff in 2.8; I note that I'd like to see the s3a perf improvements
& openstack fixes in there: for which I need reviewers. I don't have the spare time to
do this myself.

If you think they are useful, it helps to file tickets (or point out existing tickets), start
discussion etc w.r.t these areas in order to attract contributors.

> -likewise, DFSConfigKeys stayed in hdfs-server. I know it's tagged as @Private, but it's
long been where all the string constants for HDFS options live. Forcing users to retype them
in their own source is not only dangerous (it only encourages typos), it actually stops you
using your IDE finding out where those constants get used. 

> We do now have a set of keys in the client, HdfsClientConfigKeys, but these are still
declared as @Private. Which is a mistake for the reasons above, and because it encourages
hadoop developers to assume that they are free to make whatever changes they want to this
code, and if it breaks something, say "it was tagged as private”

If these are worth going after, please file tickets under HDFS-6200 if they don’t exist

> 1. We have to recognise that a lot of things marked @Private are in fact essential for
clients to use. Not just constants, but actual classes.
> 2. We have to look hard at @LimitedPrivate and question the legitimacy of tagging things
as so, especially anything "@InterfaceAudience.LimitedPrivate({""MapReduce"}) —because any
YARN app you write ends up needing those classes. For evidence, look at DistributedShell's
imports, and pick a few at random: NMClientAsyncImpl, ConverterUtils being easy targets.

There are existing tickets for some of these under YARN-1953 that need some developer love.

> Returning to the pending 2.8.0 release, there's a way to find out what's going to break:
build and test things against the snapshots, without waiting for the beta releases and expecting
the downstream projects to do it for you. If they don't build, that's a success: you've found
a compatibility problem to fix. If they don't test, well that's trouble —you are in finger
pointing time.

I’ve tried doing this in the past without much success. Some downstream components did pick
up RCs but a majority of them needed a release - hence my current approach.


View raw message