incubator-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [13/29] incubator-rocketmq-site git commit: Post a blog-Mastering Component Compatible Dependency
Date Sat, 24 Dec 2016 06:56:27 GMT
Post a blog-Mastering Component Compatible Dependency


Branch: refs/heads/asf-site
Commit: a18e488888ff5b65e6b55f12c1bd602c7be6b592
Parents: e1829ba
Author: yukon <>
Authored: Fri Dec 23 11:37:27 2016 +0800
Committer: yukon <>
Committed: Fri Dec 23 11:37:27 2016 +0800

----------------------------------------------------------------------                                       |   2 +- | 270 +++++++++++++++++++
 2 files changed, 271 insertions(+), 1 deletion(-)
diff --git a/ b/
index 682cb39..408f001 100644
--- a/
+++ b/
@@ -21,7 +21,7 @@ Please refer to **** for more details.
 ### How to post articles to **Blog**?
 New a .md file in rocketmq-sites/_posts/, Jekyll will finish the rest of the work.
-Please refer to **** for more details.
+Please refer to **** for more details.
 ### How to modify the navigation?
 Please refer to **_data/navigation.yml** for more details.
diff --git a/_posts/ b/_posts/
new file mode 100644
index 0000000..c39662a
--- /dev/null
+++ b/_posts/
@@ -0,0 +1,270 @@
+title: "Mastering Component Compatible Dependency"
+  - Maven
+  - Compatibility
+  - Dependency
+  - Java
+  - Maven
+# Preface
+This article mainly includes three first,I will introduce compatibility principle(more
details see [here]( briefly.followed
by a detailed elaborating about Java component compatible dependency,including the interface-oriented
programming,single component signature protection,single component compatibility protection
and multi-component compatibility compile time checking.Finally is the review and prospect,especially
about **Dependency Mediator** project.
+{% include toc %}
+# Compatibility
+what is compatibility?it is also the key to understanding the compatible dependency.Compatibility,
often catch our eyes through two forms,**binary compatibility**,that means they can run the
same executable code, typically machine code for a general purpose computer CPU.within the
framework of Release-to-Release Binary Compatibility in SOM (Forman, Conner, Danforth, and
Raper, Proceedings of OOPSLA '95), Java programming language binaries are binary compatible
under all relevant transformations that the authors identify (with some caveats with respect
to the addition of instance variables). here is a list of some important binary compatible
changes that the Java programming language supports:
+* Reimplementing existing methods, constructors, and initializers to improve performance.
+* Changing methods or constructors to return values on inputs for which they previously either
threw exceptions that normally should not occur or failed by going into an infinite loop or
causing a deadlock.
+* Adding new fields, methods, or constructors to an existing class or interface.
+* Deleting private fields, methods, or constructors of a class.
+* When an entire package is updated, deleting default (package-only) access fields, methods,
or constructors of classes and interfaces in the package.
+* Reordering the fields, methods, or constructors in an existing type declaration.
+* Moving a method upward in the class hierarchy.
+* Reordering the list of direct superinterfaces of a class or interface.
+* Inserting new class or interface types in the type hierarchy.
+Generally,java language is **upwards binary-compatible** with previous version,such as Java
SE 6 is upwards binary-compatible with J2SE 5.0 except for some minor incompatibilities.You
can get details from [here](
+Another compatibility form,we call it **source compatibility**,meaning that recompilation
is necessary.Java language does not support downward source compatibility. Such as if source
files use new language features or Java SE platform APIs, they will not be usable with an
earlier version of the Java platform. Usually it will throw similar error like this:
+java.lang.UnsupportedClassVersionError: :
+         Unsupported major.minor version 51.0
+         at java.lang.ClassLoader.defineClass1(Native Method)
+         at java.lang.ClassLoader.defineClassCond(Unknown Source)
+Java language source compatibility policy is as follows,except for any incompatibilities
listed further below:
+* Maintenance releases (such as 1.7.25, 1.7.26) do not introduce any new language features
or APIs. They will maintain source-compatibility with each other.
+* Functionality releases and major releases (such as 1.4.0,5.0,6.0,7.0,8.0) maintain upwards
but not downwards source-compatibility.
+Let's sum up before going on:
+* Document those incompatibility cases as much as possible, even some rare circumstances
and corner cases. 
+* Recommend developer to provide facilities that alert developers to the impact of changes
on pre-existing binaries that cannot be recompiled.
+* Generally speaking,Binary Compatibility problems may occur in the following situations:
+ * **Replacing the old with the new**,such as the Java platform Profiling Architecture in
the Java SE6,JVMDI was removed and JVMPI was disabled in Java SE 6.
+ * **Combinatorial resolution**,such as Non class files have been moved from rt.jar in Java
SE 6.if -Xbootclasspath:<path to rt.jar> is specified,and request any resource files
will fail since these resources now reside in a different jar file called resources.jar.
+ * **Behavior change**,such as Java SE 6 will throw an OverlappingFileLockException,if the
application attempts to lock the overlapping region other instances of FileChannel that has
been locked, in order to realize the backward compatibility, it provides disableSystemWideOverlappingFileLockCheck
system properties. again, also in Java SE 6, jar will preserve file modification dates and
times during extract, through the to achieve backward
+ * **Structure change**,this is also the key problem to solve in **Dependency Mediator**
project. generally speaking,modification and deletion belonging to incompatible operations.and
more attentions must be paid in the RPC service maintenance, more detailed description,please
refer to an [article]( I wrote before.
also you can read the the thirteenth chapter of Java language specification.
+# Component Compatible Dependency
+This article don't talk about **dependency analysis** and **dependency decoupling**,if you
care these topics,please read [Dependency Analysis and the Modularisation of Java Programs](
recommending you try to dive into open source project [Architecture Explorer](,through
bcel technology,analyzing and measuring component's dependency relationship.but in my opinion,[Jdepend](
was surely the most distinguished tool in dependency analysis and dependency decoupling
traverses Java class file directories and generates design quality metrics for each Java package,also
allows you to automatically measure the quality of a design in terms of its extensibility,
reusability, and maintainability to manage package dependencies effectively.
+Let's return,ask yourself,have you ever been disturbed by the following problems?
+* Your service depend a volatile service,volatile means your service depending neither an
abstract class nor a interface,this class update at irregular intervals,you often encountered
errors such as **NoSuchMethodError**,**NoSuchFieldError** and **NoClassDefFoundError** etc.
+* You are developing a new feature,but this feature can not backward compatible with previous
+what should you do ? How do you give a incompatible list in your release report.
+* Java language library is a superb collection of beautiful things,how do you make a rational
dependency exclusion and effective mediate various dependency problems when developing.
+It is the time to introduce the key idea - **Mediating Components Compatible Dependency**.Specifically,it
includs the following basic principles:
+* Interface-oriented programming
+* Single component signature protection
+* Single component compatibility protection
+* Multi-component compatibility compile time checking
+## Interface-oriented programming
+Interface-oriented programming,It's easier said than done.I will cite SLF4J design philosophy
explain this principle.
+Differing from commons logging,SLF4J use **static binder technology** so as to avoid notorious
[classloader problem](,every concrete log implementation
+have a class named StaticLoggerBinder and implementing SLF4J spi LoggerFactoryBinder.thus
you just depend SLF4J api,not concerned about underlying implementation.some people may asking,why
choose SLF4J not commons logging,I can also depend commons logging api but not concret implementation?it's
not performance but classloader problem.JCL discovery process relies on classloader hacks
to find the logging framework at runtime but this mechanism leads to numerous problems including
unexpected behavior,hard to debug classloading problems resulting in increased complexity.This
is nicely captured by Ceki (the author of Log4J, SLF4J and Logback) in Think again before
adopting the commons-logging API (which also mentions memory leaks problems observed with
JCL).and this is why SLF4J, which uses static bindings, has been created. Let's to
find the concrete realizations in SLF4J:
+// We need to use the name of the StaticLoggerBinder class, but we can't reference
+// the class itself.
+private static String STATIC_LOGGER_BINDER_PATH = "org/slf4j/impl/StaticLoggerBinder.class";
+ /**
+  * It is LoggerFactory's responsibility to track version changes and manage
+  * the compatibility list.
+  * It is assumed that all versions in the 1.6 are mutually compatible.
+  */
+static private final String[] API_COMPATIBILITY_LIST = new String[]{"1.6", "1.7"};
+// private constructor prevents instantiation
+private LoggerFactory() {}
+private final static void versionSanityCheck() {
+ try {
+    String requested = StaticLoggerBinder.REQUESTED_API_VERSION;
+    boolean match = false;
+    for (int i = 0; i < API_COMPATIBILITY_LIST.length; i++) {
+      if (requested.startsWith(API_COMPATIBILITY_LIST[i])) {
+          match = true;
+      }
+     }
+    if (!match) {
+"The requested version " + requested
+           + " by your slf4j binding is not compatible with "
+           + Arrays.asList(API_COMPATIBILITY_LIST).toString());
+"See " + VERSION_MISMATCH + " for further details.");
+    }
+} catch (java.lang.NoSuchFieldError nsfe) {
+ // given our large user base and SLF4J's commitment to backward
+ // compatibility, we cannot cry here. Only for implementations
+ // which willingly declare a REQUESTED_API_VERSION field do we
+ // emit compatibility warnings.
+} catch (Throwable e) {
+ // we should never reach here
+"Unexpected problem occured during version sanity check", e);
+  }
+you can use these skills at your library.just Let others depend on your interface package,detecting
certain implementation statically.
+If you have developed an old version library,naming it as version,you hope to
refactor some implementations,how to process multi-version compatibility?here is a skill from
maven dependency plugin:
+String hint = isMaven31() ? "maven31" : isMaven2x() ? "maven2" : "maven3";
+* Check the current Maven version to see if it's Maven 2.x.
+protected static boolean isMaven2x()
+   return !canFindCoreClass( "org.apache.maven.project.DependencyResolutionRequest" ); //
Maven 3 specific
+* Check the current Maven version to see if it's Maven 3.1.
+protected static boolean isMaven31()
+   return canFindCoreClass( "org.eclipse.aether.artifact.Artifact" ); // Maven 3.1 specific
+## Single component signature protection
+Secondly,Single component signature protection,you can using [Animal Sniffer project](
as well-known Guava library,Animal Sniffer provides tools to assist verifying that classes
compiled with a newer JDK/API are compatible with an older JDK/API.
+   <groupId>org.codehaus.mojo</groupId>
+   <artifactId>animal-sniffer-maven-plugin</artifactId>
+   <configuration>
+     <signature>
+       <groupId>org.codehaus.mojo.signature</groupId>
+       <artifactId>java16-sun</artifactId>
+       <version>1.0</version>
+     </signature>
+   </configuration>
+   <executions>
+     <execution>
+       <id>check-java16-sun</id>
+       <phase>test</phase>
+       <goals>
+         <goal>check</goal>
+       </goals>
+     </execution>
+   </executions>
+ </plugin>
+So from now on, when you performing a mvn test or mvn install operation. If there is an incompatibility
in one of the projects, you would find some output like this:
+[INFO] [animal-sniffer:check {execution: check-java-api}]
+[INFO] Checking unresolved references to org.codehaus.mojo.signature:java16-sun:1.0
+[ERROR] Undefined reference: java/lang/String.contains(Ljava/lang/CharSequence;)Z in /home/admin/workspace/kafka/target/classes/org/kafka/tools...Producer.class
+More details please see [here](
+## Single component compatibility protection
+If your a SOA develper,you almost concerned with single component compatibility protection.[Clirr
project] ( may be your best choice.Clirr is a tool that checks
Java libraries for binary and source compatibility with older releases. Basically you give
it two sets of jar files and Clirr dumps out a list of changes in the public api. The Clirr
Ant task can be configured to break the build if it detects incompatible api changes. In a
continuous integration process Clirr can automatically prevent accidental introduction of
binary or source compatibility problems.
+How to use it?just add some config snippets in your pom :
+  <groupId>org.codehaus.mojo</groupId>
+  <artifactId>clirr-maven-plugin</artifactId>
+  <version>2.6.1</version>
+then,run `mvn clirr:check`,output may be like this:
+[INFO] --- clirr-maven-plugin:2.6.1:check (default-cli) @ cmq.common ---
+[INFO] artifact checking for updates from b2bmirror-all
+[INFO] Comparing to version: 2.0.1
+[ERROR] 7005: Parameter 1 of 'public void onCompletion('
has changed its type to
+[ERROR] 7005: Parameter 1 of 'public void onException(,
java.lang.Exception)' has changed its type to
+[ERROR] 7005: Parameter 1 of 'public
send(' has changed its type to
+[ERROR] 7005: Parameter 1 of 'public
send(,' has changed
its type to
+Some inspiration in **Dependency Mediator** project just come from here.more topics about
compatible,you can reference from series article [Evolving Java-based APIs](
+## Multi-component compatibility compile time checking
+Last but not least,Multi-component compatibility compile time's the time to formally
recommend project **Dependency Mediator**.
+unlike karaf and other lightness modular technology(like osgi,class names do not need to
be unique,but the combination of class names and their defining ClassLoader must to be unique),**Dependency
Mediator** try to remedy component dependency conflicting problem before the runtime rather
than using customized [classLoader](
to agree with the coexistence of different version components,thus avoid some well-konwn errors,such
as **NoSuchMethodError**,**NoSuchFieldError** and **NoClassDefFoundError** etc.
+What is the principle of **Dependency Mediator**?**Dependency Mediator** try to mediate
various component conflicting problems.It uses jar or class as the smallest component unit,that
is ComponentEntry:
+public class ComponentEntry implements Comparable<ComponentEntry> {
+  /**
+   * Similar file name
+   */
+  private String   pathName;
+  /**
+   * Component name ,such as fully-qualified class name or jar name
+   */
+  private String   name;
+  /**
+   * Name of the jar which contains this entry,this field may be null
+   */
+  private String   jarName;
+  /**
+   * Jar entry meta info,if it is a jar file,this field may be null
+   */
+  private JarEntry entry;
+  ....
+It could scan directory(also including classpath,if you set system property scanClasspath)
and POM.if it happen to jar,but you setting checkJars to false,it could detect inner MANIFEST
file,using **Build-Jdk** and **Built-By** properties to decide whether duplicated jars.thus,output
only report whether including duplicated jar.but if you 
+setting checkJars to true or nothing to do with this property,it will further analyzes whether
existing incompatible cases.
+It also provides a maven plugin,if your project is a web project,you must run mvn insall
before next step. Maven plugin also comply with the latest annotation mojo api rather than
version 2.x.x doclet api.
+**Dependency Mediator** take advantage of **nearest definition** dependency mediation strategy.
this determines what version of a dependency will be used when multiple versions of an artifact
are encountered. **nearest definition**,means that it will use the version of the closest
dependency to your project in the tree of dependencies. You can always guarantee a version
by declaring it explicitly in your project's POM. Note that if two dependency versions are
at the same depth in the dependency tree, until Maven 2.0.8 it was not defined which one would
win, but since Maven 2.0.9 it's the order in the declaration that counts: the first declaration
+**nearest definition** means that the version used will be the closest one to your project
in the tree of dependencies, eg. if dependencies for A, B, and C are defined as A -> B
-> C -> D 2.0 and A -> E -> D 1.0, then D 1.0 will be used when building A because
the path from A to D through E is shorter. You could explicitly add a dependency to D 2.0
in A to force the use of D 2.0.
+If you consider the dependency hierarchy as a binary tree structure,Undoubtedly,**nearest
definition** means the **minimum depth**. so if you put some higher version dependency library
after lower version one(may be transitive dependency),you will face incompatible conflicting
problems.Such as :
+[WARNING] Founded conflicting dependency component:org.apache.thrift:libthrift:jar Resolved
version is org.apache.thrift:libthrift:jar:0.8.0:compile But found conflicting artifact org.apache.thrift:libthrift:0.9.1
+In this case,you could fix this problem through place libthrift 0.9.1 before the library
imported through transitive dependency.
+of course,all the establishment must meet a premise,that is:
+* **Functionality releases and major releases(such as 1.4.x, 1.5.x,2.x.x,3.x.x) have source
compatibility problems.**
+In **Dependency Mediator** 2.0 version,you could customize version differentiation strategy,not
limited the the way, also in 2.0,I will improvement compatible dependency
mediator, please wait and see.
+# Summary
+Through this article, you almost calculated the evolution process of the **Dependency Mediator** my opinion,integration with the maven enforcer plugin may be a better choice.After
all,I hope **Dependency Mediator** can keep sustainable development.I would try and donate
this project to codehaus in the near future.
+If you have better idea or improving suggestion, please contact <>.

View raw message