arrow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From w...@apache.org
Subject arrow git commit: ARROW-1379: [Java] adding maven-dependency-plugin and fixing all reported dependency errors
Date Sat, 26 Aug 2017 20:01:20 GMT
Repository: arrow
Updated Branches:
  refs/heads/master a3607d2a4 -> ea503a5f0


ARROW-1379: [Java] adding maven-dependency-plugin and fixing all reported dependency errors

There is couple of dependency issues in the current maven config. This is then leaking into
the integrating project which then needs to specify foreign dependencies just because arrow
doesn't list them properly or is pulling unnecessary dependencies just because arrow lists
them improperly.

* ```arrow-format```
```
[WARNING] Unused declared dependencies found:
[WARNING]    org.slf4j:slf4j-api:jar:1.7.25:compile
[WARNING]    com.vlkan:flatbuffers:jar:1.2.0-3f79e055:compile
[WARNING]    io.netty:netty-handler:jar:4.0.49.Final:compile
[WARNING]    com.google.guava:guava:jar:18.0:compile
```
* ```arrow-memory```
```
[WARNING] Used undeclared dependencies found:
[WARNING]    io.netty:netty-buffer:jar:4.0.49.Final:compile
[WARNING]    io.netty:netty-common:jar:4.0.49.Final:compile
[WARNING] Unused declared dependencies found:
[WARNING]    com.carrotsearch:hppc:jar:0.7.2:compile
[WARNING]    io.netty:netty-handler:jar:4.0.49.Final:compile
```
* ```arrow-tools```
```
[WARNING] Used undeclared dependencies found:
[WARNING]    com.fasterxml.jackson.core:jackson-databind:jar:2.7.9:compile
[WARNING]    com.fasterxml.jackson.core:jackson-core:jar:2.7.9:compile
[WARNING] Unused declared dependencies found:
[WARNING]    org.apache.commons:commons-lang3:jar:3.6:compile
[WARNING]    org.apache.arrow:arrow-format:jar:0.7.0-SNAPSHOT:compile
[WARNING]    io.netty:netty-handler:jar:4.0.49.Final:compile
```
* ```arrow-vector```
```
[WARNING] Used undeclared dependencies found:
[WARNING]    com.google.code.findbugs:jsr305:jar:3.0.2:compile
[WARNING]    com.vlkan:flatbuffers:jar:1.2.0-3f79e055:compile
[WARNING]    io.netty:netty-common:jar:4.0.49.Final:compile
[WARNING]    io.netty:netty-buffer:jar:4.0.49.Final:compile
[WARNING]    com.fasterxml.jackson.core:jackson-core:jar:2.7.9:compile
[WARNING] Unused declared dependencies found:
[WARNING]    org.apache.commons:commons-lang3:jar:3.6:compile
[WARNING]    io.netty:netty-handler:jar:4.0.49.Final:compile
```

I am proposing this PR to:
1. Add maven-dependency-plugin to enforce all dependencies are always listed corrctly
2. Fixing all the current dependency issues

Author: Antony Mayi <antonymayi@yahoo.com>
Author: Stepan Kadlec <stepan.kadlec@oracle.com>

Closes #978 from antonymayi/master and squashes the following commits:

d7f081ea [Antony Mayi] moving `copy-flatc` to initialize phase and `analyze` execution to
parent pom
ec727175 [Antony Mayi] removing unused apache.commons.lang3, fixing pom
8cbfe5f4 [Antony Mayi] maven-dependency-plugin: ignoring dependencies of generated sources
in arrow-vector
dc833bb3 [Stepan Kadlec] adding maven-dependency-plugin and fixing all reported dependency
errors


Project: http://git-wip-us.apache.org/repos/asf/arrow/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/ea503a5f
Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/ea503a5f
Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/ea503a5f

Branch: refs/heads/master
Commit: ea503a5f020ae48528348bb8cd0eb0635d40c52b
Parents: a3607d2
Author: Antony Mayi <antonymayi@yahoo.com>
Authored: Sat Aug 26 16:01:15 2017 -0400
Committer: Wes McKinney <wes.mckinney@twosigma.com>
Committed: Sat Aug 26 16:01:15 2017 -0400

----------------------------------------------------------------------
 java/format/pom.xml                             |   7 +-
 java/memory/pom.xml                             |  22 ++-
 java/pom.xml                                    | 191 ++++++++++++-------
 java/tools/pom.xml                              |  24 ++-
 java/vector/pom.xml                             |  35 +++-
 .../src/main/codegen/includes/vv_imports.ftl    |   2 -
 6 files changed, 183 insertions(+), 98 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/arrow/blob/ea503a5f/java/format/pom.xml
----------------------------------------------------------------------
diff --git a/java/format/pom.xml b/java/format/pom.xml
index 0e43d5e..647b716 100644
--- a/java/format/pom.xml
+++ b/java/format/pom.xml
@@ -23,12 +23,9 @@
 <name>Arrow Format</name>
 
   <properties>
-    <fbs.version>1.2.0-3f79e055</fbs.version>
     <flatc.download.skip>false</flatc.download.skip>
     <flatc.executable>${project.build.directory}/flatc-${os.detected.classifier}-${fbs.version}.exe</flatc.executable>
     <flatc.generated.files>${project.build.directory}/generated-sources/flatc</flatc.generated.files>
-    <maven-compiler-plugin.version>3.3</maven-compiler-plugin.version>
-    <maven-dependency-plugin.version>2.10</maven-dependency-plugin.version>
     <os-maven-plugin.version>1.5.0.Final</os-maven-plugin.version>
   </properties>
 
@@ -36,7 +33,6 @@
     <dependency>
       <groupId>com.vlkan</groupId>
       <artifactId>flatbuffers</artifactId>
-      <version>${fbs.version}</version>
     </dependency>
   </dependencies>
 
@@ -54,11 +50,10 @@
     <plugin> <!-- download the flatbuffer compiler -->
       <groupId>org.apache.maven.plugins</groupId>
       <artifactId>maven-dependency-plugin</artifactId>
-      <version>${maven-dependency-plugin.version}</version>
       <executions>
         <execution>
           <id>copy-flatc</id>
-          <phase>generate-sources</phase>
+          <phase>initialize</phase>
           <goals>
             <goal>copy</goal>
           </goals>

http://git-wip-us.apache.org/repos/asf/arrow/blob/ea503a5f/java/memory/pom.xml
----------------------------------------------------------------------
diff --git a/java/memory/pom.xml b/java/memory/pom.xml
index 765013b..1c4eef7 100644
--- a/java/memory/pom.xml
+++ b/java/memory/pom.xml
@@ -23,20 +23,26 @@
     <dependency>
       <groupId>com.google.code.findbugs</groupId>
       <artifactId>jsr305</artifactId>
-      <version>3.0.2</version>
     </dependency>
-
     <dependency>
-      <groupId>com.carrotsearch</groupId>
-      <artifactId>hppc</artifactId>
-      <version>0.7.2</version>
+      <groupId>com.google.guava</groupId>
+      <artifactId>guava</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>io.netty</groupId>
+      <artifactId>netty-buffer</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>io.netty</groupId>
+      <artifactId>netty-common</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.slf4j</groupId>
+      <artifactId>slf4j-api</artifactId>
     </dependency>
   </dependencies>
 
-
   <build>
   </build>
 
-
-
 </project>

http://git-wip-us.apache.org/repos/asf/arrow/blob/ea503a5f/java/pom.xml
----------------------------------------------------------------------
diff --git a/java/pom.xml b/java/pom.xml
index cfcfcd4..c4c301d 100644
--- a/java/pom.xml
+++ b/java/pom.xml
@@ -32,9 +32,11 @@
     <dep.junit.version>4.11</dep.junit.version>
     <dep.slf4j.version>1.7.25</dep.slf4j.version>
     <dep.guava.version>18.0</dep.guava.version>
+    <dep.netty.version>4.0.49.Final</dep.netty.version>
+    <dep.jackson.version>2.7.9</dep.jackson.version>
+    <dep.hadoop.version>2.7.1</dep.hadoop.version>
+    <fbs.version>1.2.0-3f79e055</fbs.version>
     <forkCount>2</forkCount>
-    <jackson.version>2.7.9</jackson.version>
-    <hadoop.version>2.7.1</hadoop.version>
     <checkstyle.failOnViolation>false</checkstyle.failOnViolation>
   </properties>
 
@@ -271,56 +273,76 @@
         </configuration>
       </plugin>
 
-        <plugin>
-          <groupId>org.apache.maven.plugins</groupId>
-          <artifactId>maven-checkstyle-plugin</artifactId>
-          <version>2.17</version>
-          <dependencies>
-            <dependency>
-              <groupId>com.puppycrawl.tools</groupId>
-              <artifactId>checkstyle</artifactId>
-              <version>6.19</version>
-            </dependency>
-            <dependency>
-              <groupId>com.google.guava</groupId>
-              <artifactId>guava</artifactId>
-              <version>${dep.guava.version}</version>
-            </dependency>
-            <dependency>
-                <groupId>org.slf4j</groupId>
-                <artifactId>jcl-over-slf4j</artifactId>
-                <version>1.7.5</version>
-            </dependency>
-          </dependencies>
-          <executions>
-            <execution>
-              <id>validate</id>
-              <phase>validate</phase>
-              <goals>
-                <goal>check</goal>
-              </goals>
-            </execution>
-          </executions>
-          <configuration>
-            <configLocation>google_checks.xml</configLocation>
-            <encoding>UTF-8</encoding>
-            <consoleOutput>true</consoleOutput>
-            <failsOnError>${checkstyle.failOnViolation}</failsOnError>
-            <failOnViolation>${checkstyle.failOnViolation}</failOnViolation>
-            <violationSeverity>warning</violationSeverity>
-            <format>xml</format>
-            <format>html</format>
-            <outputFile>${project.build.directory}/test/checkstyle-errors.xml</outputFile>
-            <linkXRef>false</linkXRef>
-          </configuration>
-        </plugin>
-
-
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-checkstyle-plugin</artifactId>
+        <version>2.17</version>
+        <dependencies>
+          <dependency>
+            <groupId>com.puppycrawl.tools</groupId>
+            <artifactId>checkstyle</artifactId>
+            <version>6.19</version>
+          </dependency>
+          <dependency>
+            <groupId>com.google.guava</groupId>
+            <artifactId>guava</artifactId>
+            <version>${dep.guava.version}</version>
+          </dependency>
+          <dependency>
+              <groupId>org.slf4j</groupId>
+              <artifactId>jcl-over-slf4j</artifactId>
+              <version>1.7.5</version>
+          </dependency>
+        </dependencies>
+        <executions>
+          <execution>
+            <id>validate</id>
+            <phase>validate</phase>
+            <goals>
+              <goal>check</goal>
+            </goals>
+          </execution>
+        </executions>
+        <configuration>
+          <configLocation>google_checks.xml</configLocation>
+          <encoding>UTF-8</encoding>
+          <consoleOutput>true</consoleOutput>
+          <failsOnError>${checkstyle.failOnViolation}</failsOnError>
+          <failOnViolation>${checkstyle.failOnViolation}</failOnViolation>
+          <violationSeverity>warning</violationSeverity>
+          <format>xml</format>
+          <format>html</format>
+          <outputFile>${project.build.directory}/test/checkstyle-errors.xml</outputFile>
+          <linkXRef>false</linkXRef>
+        </configuration>
+      </plugin>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-dependency-plugin</artifactId>
+        <executions>
+          <execution>
+            <id>analyze</id>
+            <phase>verify</phase>
+            <goals>
+              <goal>analyze-only</goal>
+            </goals>
+            <configuration>
+              <ignoreNonCompile>true</ignoreNonCompile>
+              <failOnWarning>true</failOnWarning>
+            </configuration>
+          </execution>
+        </executions>
+      </plugin>
     </plugins>
-    <pluginManagement>
 
+    <pluginManagement>
       <plugins>
         <plugin>
+          <groupId>org.apache.maven.plugins</groupId>
+          <artifactId>maven-dependency-plugin</artifactId>
+          <version>3.0.1</version>
+        </plugin>
+        <plugin>
           <groupId>org.apache.rat</groupId>
           <artifactId>apache-rat-plugin</artifactId>
           <version>0.11</version>
@@ -449,26 +471,63 @@
       </plugins>
     </pluginManagement>
   </build>
-  <dependencies>
-
-    <dependency>
-      <groupId>io.netty</groupId>
-      <artifactId>netty-handler</artifactId>
-      <version>4.0.49.Final</version>
-    </dependency>
-
-    <dependency>
-      <groupId>com.google.guava</groupId>
-      <artifactId>guava</artifactId>
-      <version>${dep.guava.version}</version>
-    </dependency>
 
-    <dependency>
-      <groupId>org.slf4j</groupId>
-      <artifactId>slf4j-api</artifactId>
-      <version>${dep.slf4j.version}</version>
-    </dependency>
+  <dependencyManagement>
+    <dependencies>
+      <dependency>
+        <groupId>com.vlkan</groupId>
+        <artifactId>flatbuffers</artifactId>
+        <version>${fbs.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>com.google.guava</groupId>
+        <artifactId>guava</artifactId>
+        <version>${dep.guava.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>io.netty</groupId>
+        <artifactId>netty-handler</artifactId>
+        <version>${dep.netty.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>io.netty</groupId>
+        <artifactId>netty-buffer</artifactId>
+        <version>${dep.netty.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>io.netty</groupId>
+        <artifactId>netty-common</artifactId>
+        <version>${dep.netty.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>com.fasterxml.jackson.core</groupId>
+        <artifactId>jackson-core</artifactId>
+        <version>${dep.jackson.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>com.fasterxml.jackson.core</groupId>
+        <artifactId>jackson-annotations</artifactId>
+        <version>${dep.jackson.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>com.fasterxml.jackson.core</groupId>
+        <artifactId>jackson-databind</artifactId>
+        <version>${dep.jackson.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>com.google.code.findbugs</groupId>
+        <artifactId>jsr305</artifactId>
+        <version>3.0.2</version>
+      </dependency>
+      <dependency>
+        <groupId>org.slf4j</groupId>
+        <artifactId>slf4j-api</artifactId>
+        <version>${dep.slf4j.version}</version>
+      </dependency>
+    </dependencies>
+  </dependencyManagement>
 
+  <dependencies>
     <!-- Test Dependencies -->
     <dependency>
       <groupId>org.slf4j</groupId>

http://git-wip-us.apache.org/repos/asf/arrow/blob/ea503a5f/java/tools/pom.xml
----------------------------------------------------------------------
diff --git a/java/tools/pom.xml b/java/tools/pom.xml
index e2e3f81..48b507a 100644
--- a/java/tools/pom.xml
+++ b/java/tools/pom.xml
@@ -22,11 +22,6 @@
     <dependencies>
         <dependency>
             <groupId>org.apache.arrow</groupId>
-            <artifactId>arrow-format</artifactId>
-            <version>${project.version}</version>
-        </dependency>
-        <dependency>
-            <groupId>org.apache.arrow</groupId>
             <artifactId>arrow-memory</artifactId>
             <version>${project.version}</version>
         </dependency>
@@ -36,9 +31,8 @@
             <version>${project.version}</version>
         </dependency>
         <dependency>
-            <groupId>org.apache.commons</groupId>
-            <artifactId>commons-lang3</artifactId>
-            <version>3.6</version>
+          <groupId>com.google.guava</groupId>
+          <artifactId>guava</artifactId>
         </dependency>
         <dependency>
             <groupId>commons-cli</groupId>
@@ -49,7 +43,19 @@
           <groupId>ch.qos.logback</groupId>
           <artifactId>logback-classic</artifactId>
           <version>1.2.3</version>
-          <scope>run</scope>
+          <scope>runtime</scope>
+        </dependency>
+        <dependency>
+          <groupId>com.fasterxml.jackson.core</groupId>
+          <artifactId>jackson-core</artifactId>
+        </dependency>
+        <dependency>
+          <groupId>com.fasterxml.jackson.core</groupId>
+          <artifactId>jackson-databind</artifactId>
+        </dependency>
+        <dependency>
+          <groupId>org.slf4j</groupId>
+          <artifactId>slf4j-api</artifactId>
         </dependency>
     </dependencies>
 

http://git-wip-us.apache.org/repos/asf/arrow/blob/ea503a5f/java/vector/pom.xml
----------------------------------------------------------------------
diff --git a/java/vector/pom.xml b/java/vector/pom.xml
index a89742e..bf33823 100644
--- a/java/vector/pom.xml
+++ b/java/vector/pom.xml
@@ -38,13 +38,15 @@
     </dependency>
     <dependency>
       <groupId>com.fasterxml.jackson.core</groupId>
+      <artifactId>jackson-core</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>com.fasterxml.jackson.core</groupId>
       <artifactId>jackson-annotations</artifactId>
-      <version>${jackson.version}</version>
     </dependency>
     <dependency>
       <groupId>com.fasterxml.jackson.core</groupId>
       <artifactId>jackson-databind</artifactId>
-      <version>${jackson.version}</version>
     </dependency>
     <dependency>
       <groupId>com.carrotsearch</groupId>
@@ -52,15 +54,34 @@
       <version>0.7.2</version>
     </dependency>
     <dependency>
-      <groupId>org.apache.commons</groupId>
-      <artifactId>commons-lang3</artifactId>
-      <version>3.6</version>
-    </dependency>
-    <dependency>
       <groupId>commons-codec</groupId>
       <artifactId>commons-codec</artifactId>
       <version>1.10</version>
     </dependency>
+    <dependency>
+      <groupId>com.google.guava</groupId>
+      <artifactId>guava</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>io.netty</groupId>
+      <artifactId>netty-buffer</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>io.netty</groupId>
+      <artifactId>netty-common</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>com.vlkan</groupId>
+      <artifactId>flatbuffers</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>com.google.code.findbugs</groupId>
+      <artifactId>jsr305</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.slf4j</groupId>
+      <artifactId>slf4j-api</artifactId>
+    </dependency>
   </dependencies>
 
     <pluginRepositories>

http://git-wip-us.apache.org/repos/asf/arrow/blob/ea503a5f/java/vector/src/main/codegen/includes/vv_imports.ftl
----------------------------------------------------------------------
diff --git a/java/vector/src/main/codegen/includes/vv_imports.ftl b/java/vector/src/main/codegen/includes/vv_imports.ftl
index e723e7d..a55304d 100644
--- a/java/vector/src/main/codegen/includes/vv_imports.ftl
+++ b/java/vector/src/main/codegen/includes/vv_imports.ftl
@@ -22,8 +22,6 @@ import com.google.flatbuffers.FlatBufferBuilder;
 import com.google.common.base.Preconditions;
 import io.netty.buffer.*;
 
-import org.apache.commons.lang3.ArrayUtils;
-
 import org.apache.arrow.memory.*;
 import org.apache.arrow.vector.types.Types;
 import org.apache.arrow.vector.types.Types.*;


Mime
View raw message