sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Cecho" <jar...@apache.org>
Subject Re: Review Request 15589: SQOOP-1232 Sqoop2: Provide tooling infrastructure for Sqoop2
Date Mon, 02 Dec 2013 17:13:25 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Dec. 2, 2013, 5:13 p.m.)

Review request for Sqoop.


Incorporated Abe's feedback.

Bugs: SQOOP-1232

Repository: sqoop-sqoop2


I've provided basic infrastructure for the tooling by adding new maven module sqoop-tools
that is depending on the core module. I've also provided example tool called "verify" that
will try to bootstrap all sqoop components to see if they are configured correctly.

I was having difficulties to make run the code with the same environment as the Sqoop 2 server
itself, but without the need to open the Tomcat for outside connections. I finally ended up
using Tool mechanism that is exposed by Tomcat. Sadly this mechanism won't set up the environment
exactly as we need it - common.loader and webapp class loaders are not loaded by this mechanism.
so I've provided a class that will finish configuring the environment and call the usual ToolRunner.
To keep the footprint of this class/jar file to a bare minimum I've placed it into standalone
maven module that do not depend on any Sqoop libraries.

Diffs (updated)

  dist/pom.xml 9186a38b3b9c57a62731f01839a99dc7cad9fc99 
  dist/src/main/bin/sqoop.sh e3ed5ef4bbaba0c3048cb005d4e183e66768117e 
  pom.xml a4bc085ce808db912b6ca516b51c70529bd081e7 
  server/pom.xml a07ecf41c2aa687aa4a2a60367e1d1265610b7e9 
  tomcat/pom.xml PRE-CREATION 
  tomcat/src/main/java/org/apache/sqoop/tomcat/TomcatToolRunner.java PRE-CREATION 
  tools/pom.xml PRE-CREATION 
  tools/src/main/java/org/apache/sqoop/tools/Tool.java PRE-CREATION 
  tools/src/main/java/org/apache/sqoop/tools/ToolRunner.java PRE-CREATION 
  tools/src/main/java/org/apache/sqoop/tools/tool/BuiltinTools.java PRE-CREATION 
  tools/src/main/java/org/apache/sqoop/tools/tool/VerifyTool.java PRE-CREATION 

Diff: https://reviews.apache.org/r/15589/diff/


I'm currently not attaching any unit tests as this task should be better covered some sort
of integration testing. I did however tested the provided verification tool on a real cluster
both with correct configuration and miss configuration (missing Hadoop dependencies).


Jarek Cecho

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message