hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hitesh Shah <hit...@apache.org>
Subject [DISCUSS] Treating LimitedPrivate({"MapReduce"}) as Public APIs for YARN applications
Date Tue, 10 May 2016 15:37:26 GMT
There have been various discussions on various JIRAs where upstream projects such as YARN apps
( Tez, Slider, etc ) are called out for using the above so-called Private APIs. A lot of YARN
applications that have been built out have picked up various bits and pieces of implementation
from MapReduce and DistributedShell to get things to work.

A recent example is a backward incompatible change introduced ( where the API is not even
directly invoked ) in the Credentials class related to the ability to read tokens/credentials
from a file. This functionality is required by pretty much everyone as YARN provides the credentials
to the app by writing the credentials/tokens to a local file which is read in when UserGroupInformation.getCurrentUser()
is invoked.  This change breaks rolling upgrades for yarn applications from 2.x to 3.x (whether
we end up supporting rolling upgrades across 2.x to 3.x is a separate discussion )

I would like to change our compatibility docs to state that any API that is marked as LimitedPrivate{Mapreduce}
implies LimitedPrivate{YARN Applications}.


— Hitesh
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

View raw message