hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjay Radia <sra...@yahoo-inc.com>
Subject Hadoop 1.0 Interface Classification Discussion
Date Fri, 14 Nov 2008 18:13:54 GMT
One of the tasks/prerequisites  for Hadoop 1.0 is to decide which  
interfaces are
  external facing and which are internal facing.

   An interface declared java-public is not necessarily a hadoop
public/external interface.  Instead I use the terms external and  
internal to
avoid confusion with a java class's public and private tag.
What does external and internal mean: Any application can use Hadoop's  
external interface;
however, they should not use any hadoop internal interfaces.

   Further it is helpful to classify the stability levels of interfaces.
   I have used a interface taxonomy that I have derived from OpenSolaris


Release terminology used below:
Standard release numbering:
- Only bug fixes in dot releases: m.x.y
	- no changes to API, disk format, protocols or config etc.
- new features in major (m.0) and minor (m.x.0) releases

    Here is a *strawman* breakdown of our various interfaces along  
with their
    visibility and stability.
    There is sure to be debate over this. Is this classification useful?
    What interfaces are not covered below?

    * External-stable (can break compatibility at major releases)
       - Filesystem, MapReduce, Config, shell CLI and shell CLI output
       - Mapred.lib (stable after cleanup)
       - HDFS-JMX MBeans?

    * External-evolving (can break compatibility at minor releases)
		(Evolving does not mean that we will break compatibility for trivial  
          at a minor release; we have the freedom to break compatibility
          when there is good reason.)
       - Core APIs that we are not ready to declare as stable: RPC and  
IPC APIs , Metrics,
              - alternatively classify  these core APIs as Internal- 
       - Job logs - should this be stable ?

    * External-Not an interface
       - Hadoop Web GUI

    * Internal-stable - (can break compatibility at major releases)
       - Intra HDFS and MR protocols (rolling upgrades in the future)

    * Internal-evolving (can break compatibility at minor releases)
       - TT-to-task umbilical cord
    * Internal-private
        - System logs, etc.
        - The rest of the Hadoop classes and interfaces.

We need to remove all internal interfaces from the hadoop-external  
     Add javadoc tag to interfaces to reflect stability? 

View raw message