atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhan Neethiraj <mad...@apache.org>
Subject Re: Review Request 49585: ATLAS-987: load dependent libraries of Atlas hooks in a separate class loader to avoid adding them to component CLASSPATH
Date Mon, 04 Jul 2016 08:34:00 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49585/
-----------------------------------------------------------

(Updated July 4, 2016, 8:33 a.m.)


Review request for atlas.


Changes
-------

- added hadoop-common, json, log4j libraries in all hooks, as they are used in AtlasHook base
class
- added missing StromAtlasHook constructor


Bugs: ATLAS-987
    https://issues.apache.org/jira/browse/ATLAS-987


Repository: atlas


Description
-------

Dependent libraries of Atlas hooks are loaded in a separate class loader, there by making
these libraries not visibile to components.

Here is the summary of the changes:
  - current contents of atlas/hook/<component> directiries, including the hook class
implementation, are moved under atlas/hook/<component>/atlas-<component>-plugin-impl
directory
  - a new component specific shim library, that includes the hook class registered with the
component, is placed in directory atlas/hook/<component>
  - the hook class in the shim library loads all files in atlas-<component>-plugin-impl
directory in a classloader and forwards all the calls on the shim class to the real implementation
class

This implementation is same as the one used in Apache Ranger - more details in RANGER-586.


Diffs (updated)
-----

  addons/falcon-bridge-shim/pom.xml PRE-CREATION 
  addons/falcon-bridge-shim/src/main/java/org/apache/atlas/falcon/hook/AtlasService.java PRE-CREATION

  addons/falcon-bridge/pom.xml d79dda9 
  addons/hive-bridge-shim/pom.xml PRE-CREATION 
  addons/hive-bridge-shim/src/main/java/org/apache/atlas/hive/hook/HiveHook.java PRE-CREATION

  addons/hive-bridge/pom.xml ddefdc2 
  addons/sqoop-bridge-shim/pom.xml PRE-CREATION 
  addons/sqoop-bridge-shim/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java PRE-CREATION

  addons/sqoop-bridge/pom.xml c792945 
  addons/storm-bridge-shim/pom.xml PRE-CREATION 
  addons/storm-bridge-shim/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java PRE-CREATION

  addons/storm-bridge/pom.xml 9e8bf2f 
  plugin-classloader/pom.xml PRE-CREATION 
  plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoader.java
PRE-CREATION 
  plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoaderUtil.java
PRE-CREATION 
  pom.xml f119525 

Diff: https://reviews.apache.org/r/49585/diff/


Testing
-------

Verified that existing tests pass
Verified Hive hook successfully sends notification to Kafka, which in turn was processed correctly
by Atlas server


Thanks,

Madhan Neethiraj


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message