flume-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hshreedha...@apache.org
Subject git commit: FLUME-2185. Upgrade morphlines to 0.7.0
Date Mon, 16 Sep 2013 22:39:34 GMT
Updated Branches:
  refs/heads/trunk 39b6b5282 -> 629b7e6b8


FLUME-2185. Upgrade morphlines to 0.7.0

(Wolfgang Hoschek via Hari Shreedharan)


Project: http://git-wip-us.apache.org/repos/asf/flume/repo
Commit: http://git-wip-us.apache.org/repos/asf/flume/commit/629b7e6b
Tree: http://git-wip-us.apache.org/repos/asf/flume/tree/629b7e6b
Diff: http://git-wip-us.apache.org/repos/asf/flume/diff/629b7e6b

Branch: refs/heads/trunk
Commit: 629b7e6b888190ebd5b9931caeec90c5d1c2ba58
Parents: 39b6b52
Author: Hari Shreedharan <hshreedharan@apache.org>
Authored: Mon Sep 16 15:38:13 2013 -0700
Committer: Hari Shreedharan <hshreedharan@apache.org>
Committed: Mon Sep 16 15:38:13 2013 -0700

----------------------------------------------------------------------
 flume-ng-doc/sphinx/FlumeUserGuide.rst          |  4 ++--
 .../flume-ng-morphline-solr-sink/pom.xml        | 20 +-------------------
 2 files changed, 3 insertions(+), 21 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/flume/blob/629b7e6b/flume-ng-doc/sphinx/FlumeUserGuide.rst
----------------------------------------------------------------------
diff --git a/flume-ng-doc/sphinx/FlumeUserGuide.rst b/flume-ng-doc/sphinx/FlumeUserGuide.rst
index c614991..bbfb5d0 100644
--- a/flume-ng-doc/sphinx/FlumeUserGuide.rst
+++ b/flume-ng-doc/sphinx/FlumeUserGuide.rst
@@ -1839,7 +1839,7 @@ This sink extracts data from Flume events, transforms it, and loads
it in near-r
 
 This sink is well suited for use cases that stream raw data into HDFS (via the HdfsSink)
and simultaneously extract, transform and load the same data into Solr (via MorphlineSolrSink).
In particular, this sink can process arbitrary heterogeneous raw data from disparate data
sources and turn it into a data model that is useful to Search applications.
 
-The ETL functionality is customizable using a `morphline configuration file <http://cloudera.github.io/cdk/docs/0.4.1/cdk-morphlines/index.html>`_
that defines a chain of transformation commands that pipe event records from one command to
another. 
+The ETL functionality is customizable using a `morphline configuration file <http://cloudera.github.io/cdk/docs/current/cdk-morphlines/index.html>`_
that defines a chain of transformation commands that pipe event records from one command to
another. 
 
 Morphlines can be seen as an evolution of Unix pipelines where the data model is generalized
to work with streams of generic records, including arbitrary binary payloads. A morphline
command is a bit like a Flume Interceptor. Morphlines can be embedded into Hadoop components
such as Flume.
 
@@ -2599,7 +2599,7 @@ prefix            ""       The prefix string constant to prepend to
each generat
 Morphline Interceptor
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-This interceptor filters the events through a `morphline configuration file <http://cloudera.github.io/cdk/docs/0.4.1/cdk-morphlines/index.html>`_
that defines a chain of transformation commands that pipe records from one command to another.
+This interceptor filters the events through a `morphline configuration file <http://cloudera.github.io/cdk/docs/current/cdk-morphlines/index.html>`_
that defines a chain of transformation commands that pipe records from one command to another.
 For example the morphline can ignore certain events or alter or insert certain event headers
via regular expression based pattern matching, or it can auto-detect and set a MIME type via
Apache Tika on events that are intercepted. For example, this kind of packet sniffing can
be used for content based dynamic routing in a Flume topology.
 MorphlineInterceptor can also help to implement dynamic routing to multiple Apache Solr collections
(e.g. for multi-tenancy).
 

http://git-wip-us.apache.org/repos/asf/flume/blob/629b7e6b/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml b/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
index a2fb931..b2640d9 100644
--- a/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
@@ -33,8 +33,7 @@ limitations under the License.
     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
     <solr.version>4.3.0</solr.version>
     <solr.expected.version>4.3.0</solr.expected.version> <!-- sanity check
to verify we actually run against the expected version rather than some outdated version -->
-    <tika.version>1.3</tika.version>
-    <cdk.version>0.6.0</cdk.version>
+    <cdk.version>0.7.0</cdk.version>
     <slf4j.version>1.6.1</slf4j.version>
     <surefire.version>2.12.4</surefire.version>
   </properties>
@@ -108,23 +107,6 @@ limitations under the License.
       </exclusions>
     </dependency>
 
-    <dependency> <!-- see http://tika.apache.org -->
-      <groupId>org.apache.tika</groupId>
-      <artifactId>tika-xmp</artifactId>
-      <version>${tika.version}</version>
-      <scope>test</scope>
-      <exclusions>
-        <exclusion>
-          <groupId>org.apache.geronimo.specs</groupId>
-          <artifactId>geronimo-stax-api_1.0_spec</artifactId> <!-- needed
by tika-parsers but already provided by JDK -->
-        </exclusion>
-        <exclusion>
-          <groupId>xerces</groupId>
-          <artifactId>xercesImpl</artifactId> <!-- used by com.drewnoakes:metadata-extractor:jar
but replacing built-in XML parser with legacy xerces is scary and probably don't need it -->
-        </exclusion>
-      </exclusions>
-    </dependency>
-
     <dependency>
       <groupId>com.cloudera.cdk</groupId>
       <artifactId>cdk-morphlines-solr-core</artifactId>


Mime
View raw message