Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B1F82200C61 for ; Tue, 21 Mar 2017 01:16:13 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id AD417160BA3; Tue, 21 Mar 2017 00:16:13 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 5ABA6160B95 for ; Tue, 21 Mar 2017 01:16:11 +0100 (CET) Received: (qmail 5509 invoked by uid 500); 21 Mar 2017 00:16:10 -0000 Mailing-List: contact commits-help@kudu.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kudu.apache.org Delivered-To: mailing list commits@kudu.apache.org Received: (qmail 5411 invoked by uid 99); 21 Mar 2017 00:16:09 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 21 Mar 2017 00:16:09 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id EEEB9DFFC2; Tue, 21 Mar 2017 00:16:08 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: todd@apache.org To: commits@kudu.apache.org Date: Tue, 21 Mar 2017 00:16:11 -0000 Message-Id: <9e4eb80da480444cb7eb6659b72dffe3@git.apache.org> In-Reply-To: <495e7c5929cd4031952cd874deaef1d9@git.apache.org> References: <495e7c5929cd4031952cd874deaef1d9@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [04/51] [partial] kudu git commit: Update docs, add 1.3 release archived-at: Tue, 21 Mar 2017 00:16:13 -0000 http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/AvroKuduOperationsProducer.html ---------------------------------------------------------------------- diff --git a/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/AvroKuduOperationsProducer.html b/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/AvroKuduOperationsProducer.html new file mode 100644 index 0000000..1dc96cd --- /dev/null +++ b/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/AvroKuduOperationsProducer.html @@ -0,0 +1,492 @@ + + + + + + +AvroKuduOperationsProducer (Kudu 1.3.0 API) + + + + + + + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+ + +
+ + +
+ + +
+
org.apache.kudu.flume.sink
+

Class AvroKuduOperationsProducer

+
+
+
    +
  • java.lang.Object
    • +
  • +
      +
    • org.apache.kudu.flume.sink.AvroKuduOperationsProducer
      • +
    +
    • +
+
+
    +
  • +
    +
    All Implemented Interfaces:
    +
    AutoCloseable, org.apache.flume.conf.Configurable, KuduOperationsProducer
    +
    +
    +
    +
    @InterfaceAudience.Public
+ @InterfaceStability.Evolving
+public class AvroKuduOperationsProducer
+extends Object
+implements KuduOperationsProducer
    +
    An Avro serializer that generates one operation per event by deserializing the event + body as an Avro record and mapping its fields to columns in a Kudu table. + +

    Avro Kudu Operations Producer configuration parameters + + + + + + + + + + + + + + + + + +
    Property NameDefaultRequired?Description
    producer.operationupsertNoThe operation used to write events to Kudu. + Supported operations are 'insert' and 'upsert'
    producer.schemaPathNoThe location of the Avro schema file used to deserialize the Avro-encoded event bodies. + It's used whenever an event does not include its own schema. If not specified, the + schema must be specified on a per-event basis, either by url or as a literal. + Schemas must be record type.

    +
    • +
+
+
+ +
+
+
    +
  • + + + +
      +
    • + + +

      Constructor Detail

      + + + +
        +
      • +

        AvroKuduOperationsProducer

        +
        public AvroKuduOperationsProducer()
        +
        • +
      +
      • +
    + +
      +
    • + + +

      Method Detail

      + + + +
        +
      • +

        configure

        +
        public void configure(org.apache.flume.Context context)
        +
        +
        Specified by:
        +
        configure in interface org.apache.flume.conf.Configurable
        +
        +
        • +
      + + + +
        +
      • +

        initialize

        +
        public void initialize(KuduTable table)
        +
        Description copied from interface: KuduOperationsProducer
        +
        Initializes the operations producer. Called between configure and + getOperations.
        +
        +
        Specified by:
        +
        initialize in interface KuduOperationsProducer
        +
        Parameters:
        +
        table - the KuduTable used to create Kudu Operation objects
        +
        +
        • +
      + + + +
        +
      • +

        getOperations

        +
        public List<Operation> getOperations(org.apache.flume.Event event)
+                              throws org.apache.flume.FlumeException
        +
        Description copied from interface: KuduOperationsProducer
        +
        Returns the operations that should be written to Kudu as a result of this event.
        +
        +
        Specified by:
        +
        getOperations in interface KuduOperationsProducer
        +
        Parameters:
        +
        event - Event to convert to one or more Operations
        +
        Returns:
        +
        List of Operations that should be written to Kudu
        +
        Throws:
        +
        org.apache.flume.FlumeException
        +
        +
        • +
      + + + + +
      • +
    +
    • +
+
+
+ + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+ + +
+ + +
+ +

Copyright © 2017 The Apache Software Foundation. All rights reserved.

+ + http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduOperationsProducer.html ---------------------------------------------------------------------- diff --git a/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduOperationsProducer.html b/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduOperationsProducer.html new file mode 100644 index 0000000..9599cb9 --- /dev/null +++ b/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduOperationsProducer.html @@ -0,0 +1,292 @@ + + + + + + +KuduOperationsProducer (Kudu 1.3.0 API) + + + + + + + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+
    +
  • Detail: 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+
+ + +
+ + +
+
org.apache.kudu.flume.sink
+

Interface KuduOperationsProducer

+
+
+
+ +
+
+
    +
  • + +
      +
    • + + +

      Method Summary

      + + + + + + + + + + + + + + + + + + +
      All Methods Instance Methods Abstract Methods 
      Modifier and TypeMethod and Description
      voidclose() +
      Cleans up any state.
      +
      List<Operation>getOperations(org.apache.flume.Event event) +
      Returns the operations that should be written to Kudu as a result of this event.
      +
      voidinitialize(KuduTable table) +
      Initializes the operations producer.
      +
      +
        +
      • + + +

        Methods inherited from interface org.apache.flume.conf.Configurable

        +configure
        • +
      +
      • +
    +
    • +
+
+
+
    +
  • + +
      +
    • + + +

      Method Detail

      + + + +
        +
      • +

        initialize

        +
        void initialize(KuduTable table)
        +
        Initializes the operations producer. Called between configure and + getOperations.
        +
        +
        Parameters:
        +
        table - the KuduTable used to create Kudu Operation objects
        +
        +
        • +
      + + + +
        +
      • +

        getOperations

        +
        List<Operation> getOperations(org.apache.flume.Event event)
        +
        Returns the operations that should be written to Kudu as a result of this event.
        +
        +
        Parameters:
        +
        event - Event to convert to one or more Operations
        +
        Returns:
        +
        List of Operations that should be written to Kudu
        +
        +
        • +
      + + + +
        +
      • +

        close

        +
        void close()
        +
        Cleans up any state. Called when the sink is stopped.
        +
        +
        Specified by:
        +
        close in interface AutoCloseable
        +
        +
        • +
      +
      • +
    +
    • +
+
+
+ + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+
    +
  • Detail: 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+
+ + +
+ +

Copyright © 2017 The Apache Software Foundation. All rights reserved.

+ + http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduSink.html ---------------------------------------------------------------------- diff --git a/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduSink.html b/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduSink.html new file mode 100644 index 0000000..a8013d4 --- /dev/null +++ b/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduSink.html @@ -0,0 +1,414 @@ + + + + + + +KuduSink (Kudu 1.3.0 API) + + + + + + + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+ +
+ + +
+ + +
+
org.apache.kudu.flume.sink
+

Class KuduSink

+
+
+
    +
  • java.lang.Object
    • +
  • +
      +
    • org.apache.flume.sink.AbstractSink
      • +
    • +
        +
      • org.apache.kudu.flume.sink.KuduSink
        • +
      +
      • +
    +
    • +
+
+
    +
  • +
    +
    All Implemented Interfaces:
    +
    org.apache.flume.conf.Configurable, org.apache.flume.lifecycle.LifecycleAware, org.apache.flume.NamedComponent, org.apache.flume.Sink
    +
    +
    +
    +
    @InterfaceAudience.Public
+ @InterfaceStability.Evolving
+public class KuduSink
+extends org.apache.flume.sink.AbstractSink
+implements org.apache.flume.conf.Configurable
    +
    A Flume sink that reads events from a channel and writes them to a Kudu table. + +

    Flume Kudu Sink configuration parameters + + + + + + + + + + + + + + + + + + + + +
    Property NameDefaultRequired?Description
    channelYesThe name of the Flume channel to read.
    typeYesComponent name. Must be org.apache.kudu.flume.sink.KuduSink
    masterAddressesYesComma-separated list of "host:port" Kudu master addresses. + The port is optional.
    tableNameYesThe name of the Kudu table to write to.
    batchSize100NoThe maximum number of events the sink takes from the channel per transaction.
    ignoreDuplicateRowstrueNoWhether to ignore duplicate primary key errors caused by inserts.
    timeoutMillis10000NoTimeout period for Kudu write operations, in milliseconds.
    producerSimpleKuduOperationsProducerNoThe fully-qualified class name of the KuduOperationsProducer + the sink should use.
    producer.*(Varies by operations producer)Configuration properties to pass to the operations producer implementation.
    + +

    Installation + +

    After building the sink, in order to use it with Flume, place the file named + kudu-flume-sink-VERSION-jar-with-dependencies.jar in the + Flume plugins.d directory under kudu-flume-sink/lib/. + +

    For detailed instructions on using Flume's plugins.d mechanism, please see the plugins.d + section of the Flume User Guide.

    +
    • +
+
+
+ +
+
+
    +
  • + +
      +
    • + + +

      Constructor Detail

      + + + +
        +
      • +

        KuduSink

        +
        public KuduSink()
        +
        • +
      + + + +
        +
      • +

        KuduSink

        +
        @InterfaceAudience.Private
+public KuduSink(KuduClient kuduClient)
        +
        • +
      +
      • +
    + +
      +
    • + + +

      Method Detail

      + + + +
        +
      • +

        start

        +
        public void start()
        +
        +
        Specified by:
        +
        start in interface org.apache.flume.lifecycle.LifecycleAware
        +
        Overrides:
        +
        start in class org.apache.flume.sink.AbstractSink
        +
        +
        • +
      + + + +
        +
      • +

        stop

        +
        public void stop()
        +
        +
        Specified by:
        +
        stop in interface org.apache.flume.lifecycle.LifecycleAware
        +
        Overrides:
        +
        stop in class org.apache.flume.sink.AbstractSink
        +
        +
        • +
      + + + +
        +
      • +

        configure

        +
        public void configure(org.apache.flume.Context context)
        +
        +
        Specified by:
        +
        configure in interface org.apache.flume.conf.Configurable
        +
        +
        • +
      + + + +
        +
      • +

        getClient

        +
        public KuduClient getClient()
        +
        • +
      + + + +
        +
      • +

        process

        +
        public org.apache.flume.Sink.Status process()
+                                     throws org.apache.flume.EventDeliveryException
        +
        +
        Specified by:
        +
        process in interface org.apache.flume.Sink
        +
        Throws:
        +
        org.apache.flume.EventDeliveryException
        +
        +
        • +
      +
      • +
    +
    • +
+
+
+ + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+ +
+ + +
+ +

Copyright © 2017 The Apache Software Foundation. All rights reserved.

+ + http://git-wip-us.apache.org/repos/asf/kudu/blob/d114777e/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.html ---------------------------------------------------------------------- diff --git a/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.html b/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.html new file mode 100644 index 0000000..2e56801 --- /dev/null +++ b/releases/1.3.0/apidocs/org/apache/kudu/flume/sink/KuduSinkConfigurationConstants.html @@ -0,0 +1,411 @@ + + + + + + +KuduSinkConfigurationConstants (Kudu 1.3.0 API) + + + + + + + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+ + +
+ + +
+ + +
+
org.apache.kudu.flume.sink
+

Class KuduSinkConfigurationConstants

+
+
+
    +
  • java.lang.Object
    • +
  • +
      +
    • org.apache.kudu.flume.sink.KuduSinkConfigurationConstants
      • +
    +
    • +
+
+
    +
  • +
    +
    +
    @InterfaceAudience.Public
+ @InterfaceStability.Evolving
+public class KuduSinkConfigurationConstants
+extends Object
    +
    • +
+
+
+ +
+
+
    +
  • + +
      +
    • + + +

      Field Detail

      + + + +
        +
      • +

        MASTER_ADDRESSES

        +
        public static final String MASTER_ADDRESSES
        +
        Comma-separated list of "host:port" Kudu master addresses. + The port is optional and defaults to the Kudu Java client's default master + port.
        +
        +
        See Also:
        +
        Constant Field Values
        +
        +
        • +
      + + + +
        +
      • +

        TABLE_NAME

        +
        public static final String TABLE_NAME
        +
        The name of the table in Kudu to write to.
        +
        +
        See Also:
        +
        Constant Field Values
        +
        +
        • +
      + + + +
        +
      • +

        PRODUCER

        +
        public static final String PRODUCER
        +
        The fully qualified class name of the KuduOperationsProducer class that the + sink should use.
        +
        +
        See Also:
        +
        Constant Field Values
        +
        +
        • +
      + + + +
        +
      • +

        PRODUCER_PREFIX

        +
        public static final String PRODUCER_PREFIX
        +
        Prefix for configuration parameters that are passed to the + KuduOperationsProducer.
        +
        +
        See Also:
        +
        Constant Field Values
        +
        +
        • +
      + + + +
        +
      • +

        BATCH_SIZE

        +
        public static final String BATCH_SIZE
        +
        Maximum number of events that the sink should take from the channel per + transaction.
        +
        +
        See Also:
        +
        Constant Field Values
        +
        +
        • +
      + + + +
        +
      • +

        TIMEOUT_MILLIS

        +
        public static final String TIMEOUT_MILLIS
        +
        Timeout period for Kudu operations, in milliseconds.
        +
        +
        See Also:
        +
        Constant Field Values
        +
        +
        • +
      + + + +
        +
      • +

        IGNORE_DUPLICATE_ROWS

        +
        public static final String IGNORE_DUPLICATE_ROWS
        +
        Whether to ignore duplicate primary key errors caused by inserts.
        +
        +
        See Also:
        +
        Constant Field Values
        +
        +
        • +
      +
      • +
    + +
      +
    • + + +

      Constructor Detail

      + + + +
        +
      • +

        KuduSinkConfigurationConstants

        +
        public KuduSinkConfigurationConstants()
        +
        • +
      +
      • +
    +
    • +
+
+
+ + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+ + +
+ + +
+ +

Copyright © 2017 The Apache Software Foundation. All rights reserved.

+ +