fluo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mwa...@apache.org
Subject [1/2] incubator-fluo-recipes git commit: Fixes #92 - Added dependency analysis plugin
Date Thu, 21 Jul 2016 16:47:00 GMT
Repository: incubator-fluo-recipes
Updated Branches:
  refs/heads/master 9be53cba4 -> ac898df17


Fixes #92 - Added dependency analysis plugin

* Updated dependencies that caused failure by analysis plugin
* Updated README.md
* Removed commons-lang dependency in core


Project: http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/commit/65ddebaa
Tree: http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/tree/65ddebaa
Diff: http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/diff/65ddebaa

Branch: refs/heads/master
Commit: 65ddebaac109505a9e0731e6804573a1b2213960
Parents: 9be53cb
Author: Mike Walch <mwalch@gmail.com>
Authored: Mon Jul 18 17:29:30 2016 -0400
Committer: Mike Walch <mwalch@gmail.com>
Committed: Thu Jul 21 12:37:24 2016 -0400

----------------------------------------------------------------------
 README.md                                       | 50 ++++++++----
 modules/accumulo/pom.xml                        | 16 +++-
 modules/core/pom.xml                            | 23 ++++--
 .../recipes/core/export/DocumentLoader.java     |  3 +-
 modules/kryo/pom.xml                            |  9 +-
 modules/spark/pom.xml                           | 34 ++++++--
 modules/test/pom.xml                            | 14 +++-
 pom.xml                                         | 86 ++++++++++++++++++++
 8 files changed, 201 insertions(+), 34 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/blob/65ddebaa/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 024cb58..b592204 100644
--- a/README.md
+++ b/README.md
@@ -1,10 +1,17 @@
 # Apache Fluo Recipes
 
-[![Build Status](https://travis-ci.org/apache/fluo-recipes.svg?branch=master)](https://travis-ci.org/apache/fluo-recipes)
+[![Build Status][ti]][tl] [![Apache License][li]][ll]
 
-Common code for [Apache Fluo][fluo] application developers.  
+**Fluo Recipes are common code for [Apache Fluo][fluo] application developers.**
 
-### Available Recipes
+Fluo Recipes build on the [Fluo API][fluo-api] to offer additinal functionality to
+developers. They are published seperately from Fluo on their own release schedule.
+This allows Fluo Recipes to iterate and innovate faster than Fluo (which will maintain
+a more minimal API on a slower release cycle).
+
+### Documentation
+
+Recipes are documented below and in the [Recipes API docs][recipes-api].
 
 * [Collision Free Map][cfm] - A recipe for making many to many updates.
 * [Export Queue][export-q] - A recipe for exporting data from Fluo to external systems.
@@ -13,23 +20,25 @@ Common code for [Apache Fluo][fluo] application developers.
 operations to a log which can be used to export data from Fluo.
 * [Testing][testing] Some code to help write Fluo Integration test.
 
-### Common Functionality
-
 Recipes have common needs that are broken down into the following reusable components.
 
 * [Serialization][serialization] - Common code for serializing POJOs. 
 * [Transient Ranges][transient] - Standardized process for dealing with transient data.
 * [Table optimization][optimization] - Standardized process for optimizing the Fluo table.
 
-### Modules and Maven Imports
+### Usage
+
+The Fluo Recipes project publishes multiple jars to Maven Central for each release.
+The `fluo-recipes-core` jar is the primary jar. It is where most recipes live and where
+they are placed by default if they have minimal dependencies beyond the Fluo API.
 
-Fluo Recipes provides multiple jars.  The primary reason its separated
-into multiple jars is to avoid pushing unneeded dependencies on users.   For
-example there is a Fluo Recipes spark module that depends on Spark.  If an
-application does not need to use Spark, then it can easily avoid a transitive
-dependency on Spark.
+Fluo Recipes with dependencies that bring in many transitive dependencies publish
+their own jar. For example, recipes that depend on Apache Spark are published in the
+`fluo-recipes-spark` jar.  If you don't plan on using code in the `fluo-recipes-spark`
+jar, you should avoid including it in your pom.xml to avoid a transitive dependency on
+Spark.
 
-Below are Maven dependencies for Fluo Recipes.
+Below is a sample Maven POM containing all possible Fluo Recipes dependencies:
 
 ```xml
   <properties>
@@ -37,32 +46,31 @@ Below are Maven dependencies for Fluo Recipes.
   </properties>
 
   <dependencies>
-    <!-- Recipes core only has a transitive dependency on the Fluo API -->
+    <!-- Required. Contains recipes that are only depend on the Fluo API -->
     <dependency>
       <groupId>org.apache.fluo</groupId>
       <artifactId>fluo-recipes-core</artifactId>
       <version>${fluo-recipes.version}</version>
     </dependency>
-    <!--optional dependency provides the default Fluo Recipes serialization
-        mechanism, which is based on Kryo  -->
+    <!-- Optional. Serialization code that depends on Kryo -->
     <dependency>
       <groupId>org.apache.fluo</groupId>
       <artifactId>fluo-recipes-kryo</artifactId>
       <version>${fluo-recipes.version}</version>
     </dependency>
-    <!--optional dependency assist w/ intergrating Accumulo and Fluo  -->
+    <!-- Optional. Common code for using Fluo with Accumulo -->
     <dependency>
       <groupId>org.apache.fluo</groupId>
       <artifactId>fluo-recipes-accumulo</artifactId>
       <version>${fluo-recipes.version}</version>
     </dependency>
-    <!--optional dependency assist w/ intergrating Spark and Fluo -->
+    <!-- Optional. Common code for using Fluo with Spark -->
     <dependency>
       <groupId>org.apache.fluo</groupId>
       <artifactId>fluo-recipes-spark</artifactId>
       <version>${fluo-recipes.version}</version>
     </dependency>
-    <!--optional dependency helps when write Fluo intergeration test. -->
+    <!-- Optional. Common code for writing Fluo integration tests -->
     <dependency>
       <groupId>org.apache.fluo</groupId>
       <artifactId>fluo-recipes-test</artifactId>
@@ -73,6 +81,8 @@ Below are Maven dependencies for Fluo Recipes.
 ```
 
 [fluo]: https://fluo.apache.org/
+[fluo-api]: https://fluo.apache.org/apidocs/fluo/
+[recipes-api]: https://fluo.apache.org/apidocs/fluo-recipes/
 [cfm]: docs/cfm.md
 [export-q]: docs/export-queue.md
 [recording-tx]: docs/recording-tx.md
@@ -81,3 +91,7 @@ Below are Maven dependencies for Fluo Recipes.
 [optimization]: docs/table-optimization.md
 [row-hasher]: docs/row-hasher.md
 [testing]: docs/testing.md
+[ti]: https://travis-ci.org/apache/incubator-fluo-recipes.svg?branch=master
+[tl]: https://travis-ci.org/apache/incubator-fluo-recipes
+[li]: http://img.shields.io/badge/license-ASL-blue.svg
+[ll]: https://github.com/apache/incubator-fluo-recipes/blob/master/LICENSE

http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/blob/65ddebaa/modules/accumulo/pom.xml
----------------------------------------------------------------------
diff --git a/modules/accumulo/pom.xml b/modules/accumulo/pom.xml
index f16a9ff..03df1e5 100644
--- a/modules/accumulo/pom.xml
+++ b/modules/accumulo/pom.xml
@@ -22,9 +22,17 @@
     <relativePath>../../pom.xml</relativePath>
   </parent>
   <artifactId>fluo-recipes-accumulo</artifactId>
-  <name>Fluo Accumulo Recipes</name>
+  <name>Fluo Recipes Accumulo</name>
   <dependencies>
     <dependency>
+      <groupId>com.google.guava</groupId>
+      <artifactId>guava</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>commons-configuration</groupId>
+      <artifactId>commons-configuration</artifactId>
+    </dependency>
+    <dependency>
       <groupId>javax.inject</groupId>
       <artifactId>javax.inject</artifactId>
       <version>1</version>
@@ -42,11 +50,15 @@
       <artifactId>fluo-recipes-core</artifactId>
     </dependency>
     <dependency>
+      <groupId>org.slf4j</groupId>
+      <artifactId>slf4j-api</artifactId>
+    </dependency>
+    <!-- Test Dependencies -->
+    <dependency>
       <groupId>junit</groupId>
       <artifactId>junit</artifactId>
       <scope>test</scope>
     </dependency>
-    <!-- Test Dependencies -->
     <dependency>
       <groupId>org.apache.fluo</groupId>
       <artifactId>fluo-core</artifactId>

http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/blob/65ddebaa/modules/core/pom.xml
----------------------------------------------------------------------
diff --git a/modules/core/pom.xml b/modules/core/pom.xml
index 5210ac7..4d6893d 100644
--- a/modules/core/pom.xml
+++ b/modules/core/pom.xml
@@ -22,27 +22,40 @@
     <relativePath>../../pom.xml</relativePath>
   </parent>
   <artifactId>fluo-recipes-core</artifactId>
-  <name>Fluo Recipes</name>
+  <name>Fluo Recipes Core</name>
   <dependencies>
     <dependency>
-      <groupId>com.esotericsoftware</groupId>
-      <artifactId>kryo</artifactId>
-      <version>3.0.3</version>
+      <groupId>com.google.guava</groupId>
+      <artifactId>guava</artifactId>
     </dependency>
     <dependency>
       <groupId>commons-collections</groupId>
       <artifactId>commons-collections</artifactId>
     </dependency>
     <dependency>
+      <groupId>javax.xml.bind</groupId>
+      <artifactId>jaxb-api</artifactId>
+    </dependency>
+    <dependency>
       <groupId>org.apache.fluo</groupId>
       <artifactId>fluo-api</artifactId>
     </dependency>
+    <!-- Test Dependencies -->
+    <dependency>
+      <groupId>com.google.code.gson</groupId>
+      <artifactId>gson</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>commons-io</groupId>
+      <artifactId>commons-io</artifactId>
+      <scope>test</scope>
+    </dependency>
     <dependency>
       <groupId>junit</groupId>
       <artifactId>junit</artifactId>
       <scope>test</scope>
     </dependency>
-    <!-- Test Dependencies -->
     <dependency>
       <groupId>org.apache.fluo</groupId>
       <artifactId>fluo-core</artifactId>

http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/blob/65ddebaa/modules/core/src/test/java/org/apache/fluo/recipes/core/export/DocumentLoader.java
----------------------------------------------------------------------
diff --git a/modules/core/src/test/java/org/apache/fluo/recipes/core/export/DocumentLoader.java
b/modules/core/src/test/java/org/apache/fluo/recipes/core/export/DocumentLoader.java
index 5e7f224..f7af791 100644
--- a/modules/core/src/test/java/org/apache/fluo/recipes/core/export/DocumentLoader.java
+++ b/modules/core/src/test/java/org/apache/fluo/recipes/core/export/DocumentLoader.java
@@ -15,7 +15,6 @@
 
 package org.apache.fluo.recipes.core.export;
 
-import org.apache.commons.lang.StringUtils;
 import org.apache.fluo.recipes.core.types.TypedLoader;
 import org.apache.fluo.recipes.core.types.TypedTransactionBase;
 
@@ -31,6 +30,6 @@ public class DocumentLoader extends TypedLoader {
 
   @Override
   public void load(TypedTransactionBase tx, Context context) throws Exception {
-    tx.mutate().row("d:" + docid).fam("content").qual("new").set(StringUtils.join(refs, "
"));
+    tx.mutate().row("d:" + docid).fam("content").qual("new").set(String.join(" ", refs));
   }
 }

http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/blob/65ddebaa/modules/kryo/pom.xml
----------------------------------------------------------------------
diff --git a/modules/kryo/pom.xml b/modules/kryo/pom.xml
index 72228f1..fe00f96 100644
--- a/modules/kryo/pom.xml
+++ b/modules/kryo/pom.xml
@@ -27,7 +27,14 @@
     <dependency>
       <groupId>com.esotericsoftware</groupId>
       <artifactId>kryo</artifactId>
-      <version>3.0.3</version>
+    </dependency>
+    <dependency>
+      <groupId>com.google.guava</groupId>
+      <artifactId>guava</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.fluo</groupId>
+      <artifactId>fluo-api</artifactId>
     </dependency>
     <dependency>
       <groupId>org.apache.fluo</groupId>

http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/blob/65ddebaa/modules/spark/pom.xml
----------------------------------------------------------------------
diff --git a/modules/spark/pom.xml b/modules/spark/pom.xml
index 8e037c1..1a0969e 100644
--- a/modules/spark/pom.xml
+++ b/modules/spark/pom.xml
@@ -22,9 +22,13 @@
     <relativePath>../../pom.xml</relativePath>
   </parent>
   <artifactId>fluo-recipes-spark</artifactId>
-  <name>Fluo Spark Recipes</name>
+  <name>Fluo Recipes Spark</name>
   <dependencies>
     <dependency>
+      <groupId>commons-configuration</groupId>
+      <artifactId>commons-configuration</artifactId>
+    </dependency>
+    <dependency>
       <groupId>org.apache.accumulo</groupId>
       <artifactId>accumulo-core</artifactId>
     </dependency>
@@ -34,17 +38,33 @@
     </dependency>
     <dependency>
       <groupId>org.apache.fluo</groupId>
-      <artifactId>fluo-core</artifactId>
+      <artifactId>fluo-mapreduce</artifactId>
     </dependency>
     <dependency>
-      <groupId>org.apache.fluo</groupId>
-      <artifactId>fluo-mapreduce</artifactId>
+      <groupId>org.apache.hadoop</groupId>
+      <artifactId>hadoop-client</artifactId>
+      <exclusions>
+        <exclusion>
+          <!-- Causes conflicts with servlet-api in Spark -->
+          <groupId>javax.servlet</groupId>
+          <artifactId>servlet-api</artifactId>
+        </exclusion>
+      </exclusions>
     </dependency>
     <dependency>
       <groupId>org.apache.spark</groupId>
       <artifactId>spark-core_2.10</artifactId>
     </dependency>
     <dependency>
+      <groupId>org.scala-lang</groupId>
+      <artifactId>scala-library</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>org.slf4j</groupId>
+      <artifactId>slf4j-api</artifactId>
+    </dependency>
+    <!-- Test Dependencies -->
+    <dependency>
       <groupId>junit</groupId>
       <artifactId>junit</artifactId>
       <scope>test</scope>
@@ -54,7 +74,11 @@
       <artifactId>accumulo-minicluster</artifactId>
       <scope>test</scope>
     </dependency>
-    <!-- Test Dependencies -->
+    <dependency>
+      <groupId>org.apache.fluo</groupId>
+      <artifactId>fluo-core</artifactId>
+      <scope>test</scope>
+    </dependency>
     <dependency>
       <groupId>org.apache.fluo</groupId>
       <artifactId>fluo-mini</artifactId>

http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/blob/65ddebaa/modules/test/pom.xml
----------------------------------------------------------------------
diff --git a/modules/test/pom.xml b/modules/test/pom.xml
index b2d2e4a..8bfb466 100644
--- a/modules/test/pom.xml
+++ b/modules/test/pom.xml
@@ -22,9 +22,17 @@
     <relativePath>../../pom.xml</relativePath>
   </parent>
   <artifactId>fluo-recipes-test</artifactId>
-  <name>Fluo Test Recipes</name>
+  <name>Fluo Recipes Test</name>
   <dependencies>
     <dependency>
+      <groupId>com.google.guava</groupId>
+      <artifactId>guava</artifactId>
+    </dependency>
+    <dependency>
+      <groupId>commons-io</groupId>
+      <artifactId>commons-io</artifactId>
+    </dependency>
+    <dependency>
       <groupId>junit</groupId>
       <artifactId>junit</artifactId>
     </dependency>
@@ -55,6 +63,10 @@
       <groupId>org.apache.fluo</groupId>
       <artifactId>fluo-recipes-core</artifactId>
     </dependency>
+    <dependency>
+      <groupId>org.slf4j</groupId>
+      <artifactId>slf4j-api</artifactId>
+    </dependency>
     <!-- Test Dependencies -->
     <dependency>
       <groupId>org.apache.fluo</groupId>

http://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes/blob/65ddebaa/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index c13d461..9a90a1d 100644
--- a/pom.xml
+++ b/pom.xml
@@ -52,6 +52,7 @@
   </issueManagement>
   <properties>
     <accumulo.version>1.6.5</accumulo.version>
+    <curator.version>2.7.1</curator.version>
     <findbugs.maxRank>13</findbugs.maxRank>
     <fluo.version>1.0.0-incubating-SNAPSHOT</fluo.version>
     <hadoop.version>2.6.3</hadoop.version>
@@ -63,11 +64,46 @@
   <dependencyManagement>
     <dependencies>
       <dependency>
+        <groupId>com.esotericsoftware</groupId>
+        <artifactId>kryo</artifactId>
+        <version>3.0.3</version>
+      </dependency>
+      <dependency>
+        <groupId>com.google.code.gson</groupId>
+        <artifactId>gson</artifactId>
+        <version>2.2.4</version>
+      </dependency>
+      <dependency>
+        <groupId>com.google.guava</groupId>
+        <artifactId>guava</artifactId>
+        <version>13.0.1</version>
+      </dependency>
+      <dependency>
         <groupId>commons-collections</groupId>
         <artifactId>commons-collections</artifactId>
         <version>3.2.1</version>
       </dependency>
       <dependency>
+        <groupId>commons-configuration</groupId>
+        <artifactId>commons-configuration</artifactId>
+        <version>1.10</version>
+      </dependency>
+      <dependency>
+        <groupId>commons-io</groupId>
+        <artifactId>commons-io</artifactId>
+        <version>2.4</version>
+      </dependency>
+      <dependency>
+        <groupId>javax.servlet</groupId>
+        <artifactId>javax.servlet-api</artifactId>
+        <version>3.1.0</version>
+      </dependency>
+      <dependency>
+        <groupId>javax.xml.bind</groupId>
+        <artifactId>jaxb-api</artifactId>
+        <version>2.2.2</version>
+      </dependency>
+      <dependency>
         <groupId>junit</groupId>
         <artifactId>junit</artifactId>
         <version>4.12</version>
@@ -83,6 +119,21 @@
         <version>${accumulo.version}</version>
       </dependency>
       <dependency>
+        <groupId>org.apache.curator</groupId>
+        <artifactId>curator-client</artifactId>
+        <version>${curator.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.curator</groupId>
+        <artifactId>curator-framework</artifactId>
+        <version>${curator.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.curator</groupId>
+        <artifactId>curator-recipes</artifactId>
+        <version>${curator.version}</version>
+      </dependency>
+      <dependency>
         <groupId>org.apache.fluo</groupId>
         <artifactId>fluo-accumulo</artifactId>
         <version>${fluo.version}</version>
@@ -152,6 +203,16 @@
         <artifactId>easymock</artifactId>
         <version>3.4</version>
       </dependency>
+      <dependency>
+        <groupId>org.scala-lang</groupId>
+        <artifactId>scala-library</artifactId>
+        <version>2.10.4</version>
+      </dependency>
+      <dependency>
+        <groupId>org.slf4j</groupId>
+        <artifactId>slf4j-api</artifactId>
+        <version>1.7.12</version>
+      </dependency>
     </dependencies>
   </dependencyManagement>
   <repositories>
@@ -248,6 +309,31 @@
           </execution>
         </executions>
       </plugin>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-dependency-plugin</artifactId>
+        <version>2.10</version>
+        <executions>
+          <execution>
+            <id>analyze</id>
+            <goals>
+              <goal>analyze-only</goal>
+            </goals>
+            <configuration>
+              <failOnWarning>true</failOnWarning>
+              <ignoredDependencies>
+                <ignoredDependency>org.apache.accumulo:accumulo-minicluster:jar:${accumulo.version}</ignoredDependency>
+                <ignoredDependency>org.apache.fluo:fluo-core:jar:${fluo.version}</ignoredDependency>
+                <ignoredDependency>org.apache.fluo:fluo-mini:jar:${fluo.version}</ignoredDependency>
+                <ignoredDependency>org.apache.fluo:fluo-recipes-kryo:jar:${project.version}</ignoredDependency>
+                <ignoredDependency>org.apache.hadoop:hadoop-client:jar:${hadoop.version}</ignoredDependency>
+                <ignoredDependency>org.apache.hadoop:hadoop-common:jar:${hadoop.version}</ignoredDependency>
+                <ignoredDependency>org.apache.hadoop:hadoop-mapreduce-client-core:jar:${hadoop.version}</ignoredDependency>
+              </ignoredDependencies>
+            </configuration>
+          </execution>
+        </executions>
+      </plugin>
     </plugins>
   </build>
 </project>


Mime
View raw message