hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: [DISCUSS] Looking to a 2.8.0 release
Date Wed, 25 Nov 2015 19:53:16 GMT
Regarding interface visibility/stability, I'm aware of 2 relevant JIRAs
right now.

HADOOP-10776 proposes to mark Public some of the security plumbing like
the FileSystem delegation token methods, UserGroupInformation, Token and
Credentials.  At this point, I think we are only fooling ourselves trying
to treat these as Private or LimitedPrivate.  I believe they are de facto
Public, because downstream applications just don't have any other reliable
way to do what these interfaces do.

HADOOP-12600 proposes to mark FileContext Stable.  I expect this one is
simply an oversight.

I've taken the bold step of marking both issues as 2.8.0 blockers.  We can
of course reconsider if this is controversial.

--Chris Nauroth

On 11/25/15, 11:30 AM, "Vinod Kumar Vavilapalli" <vinodkv@apache.org>

>> There's a lot of stuff in 2.8; I note that I'd like to see the s3a perf
>>improvements & openstack fixes in there: for which I need reviewers. I
>>don't have the spare time to do this myself.
>If you think they are useful, it helps to file tickets (or point out
>existing tickets), start discussion etc w.r.t these areas in order to
>attract contributors.
>> -likewise, DFSConfigKeys stayed in hdfs-server. I know it's tagged as
>>@Private, but it's long been where all the string constants for HDFS
>>options live. Forcing users to retype them in their own source is not
>>only dangerous (it only encourages typos), it actually stops you using
>>your IDE finding out where those constants get used.
>> We do now have a set of keys in the client, HdfsClientConfigKeys, but
>>these are still declared as @Private. Which is a mistake for the reasons
>>above, and because it encourages hadoop developers to assume that they
>>are free to make whatever changes they want to this code, and if it
>>breaks something, say "it was tagged as private²
>If these are worth going after, please file tickets under HDFS-6200 if
>they don¹t exist already.
>> 1. We have to recognise that a lot of things marked @Private are in
>>fact essential for clients to use. Not just constants, but actual
>> 2. We have to look hard at @LimitedPrivate and question the legitimacy
>>of tagging things as so, especially anything
>>"@InterfaceAudience.LimitedPrivate({""MapReduce"}) ‹because any YARN app
>>you write ends up needing those classes. For evidence, look at
>>DistributedShell's imports, and pick a few at random: NMClientAsyncImpl,
>>ConverterUtils being easy targets.
>There are existing tickets for some of these under YARN-1953 that need
>some developer love.
>> Returning to the pending 2.8.0 release, there's a way to find out
>>what's going to break: build and test things against the snapshots,
>>without waiting for the beta releases and expecting the downstream
>>projects to do it for you. If they don't build, that's a success: you've
>>found a compatibility problem to fix. If they don't test, well that's
>>trouble ‹you are in finger pointing time.
>I¹ve tried doing this in the past without much success. Some downstream
>components did pick up RCs but a majority of them needed a release -
>hence my current approach.

View raw message