kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aw...@apache.org
Subject [kudu] branch branch-1.9.x updated: docs: KUDU-2411: Document the binary test jar
Date Thu, 07 Mar 2019 23:15:08 GMT
This is an automated email from the ASF dual-hosted git repository.

awong pushed a commit to branch branch-1.9.x
in repository https://gitbox.apache.org/repos/asf/kudu.git


The following commit(s) were added to refs/heads/branch-1.9.x by this push:
     new 6edf57d  docs: KUDU-2411: Document the binary test jar
6edf57d is described below

commit 6edf57d7cfaf0c223bf2d098c3a927390431627d
Author: Mike Percy <mpercy@apache.org>
AuthorDate: Wed Mar 6 15:59:39 2019 -0800

    docs: KUDU-2411: Document the binary test jar
    
    This patch adds documentation for how to use the Kudu binary test jar.
    
    Change-Id: I8189f1703626587a5313d8c1fb11d046455d9f39
    Reviewed-on: http://gerrit.cloudera.org:8080/12685
    Tested-by: Kudu Jenkins
    Reviewed-by: Grant Henke <granthenke@apache.org>
    (cherry picked from commit 9d6b117c4aa5be1aadf3707e429e939e6e388309)
    Reviewed-on: http://gerrit.cloudera.org:8080/12690
    Reviewed-by: Andrew Wong <awong@cloudera.com>
---
 docs/developing.adoc | 133 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 132 insertions(+), 1 deletion(-)

diff --git a/docs/developing.adoc b/docs/developing.adoc
index cda2f45..faebe31 100644
--- a/docs/developing.adoc
+++ b/docs/developing.adoc
@@ -63,7 +63,7 @@ out of date.
 
 These examples should serve as helpful starting points for your own Kudu applications and
integrations.
 
-=== Maven Artifacts
+== Maven Artifacts
 The following Maven `<dependency>` element is valid for the Apache Kudu public release
 (since 1.0.0):
 
@@ -259,6 +259,137 @@ requests from new clients.
 - Kudu tables may only be registered as temporary tables in SparkSQL.
   Kudu tables may not be queried using HiveContext.
 
+== JVM-Based Integration Testing
+
+As of version 1.9.0, Kudu ships with an experimental feature called the binary
+test JAR. This feature gives people who want to test against Kudu the
+capability to start a Kudu "mini cluster" from Java or another JVM-based
+language without having to first build Kudu locally. This is possible because
+the Kudu binary JAR contains relocatable Kudu binaries that are used by the
+`KuduTestHarness` in the `kudu-test-utils` module. The `KuduTestHarness`
+contains logic to search the classpath for the Kudu binaries and to start a
+mini cluster using them.
+
+_Important: The `kudu-binary` module should only be used to run Kudu for
+integration testing purposes. It should never be used to run an actual Kudu
+service, in production or development, because the `kudu-binary` module
+includes native security-related dependencies that have been copied from the
+build system and will not be patched when the operating system on the runtime
+host is patched._
+
+=== System Requirements
+
+The binary test JAR must be run on one of the
+<<installation.adoc#,supported Kudu platforms>>, which includes:
+
+- macOS El Capitan (10.11) or later;
+- CentOS 6.6+, Ubuntu 14.04+, or another recent distribution of Linux
+
+The related Maven integration using `os-maven-plugin` requires Maven 3.1 or later.
+
+=== Using the Kudu Binary Test Jar
+
+Take the following steps to start a Kudu mini cluster from a Java project.
+
+**1. Add build-time dependencies.** The `kudu-binary` artifact contains the
+native Kudu (server and command-line tool) binaries for specific operating
+systems. In order to download the right artifact for the running operating
+system, use the `os-maven-plugin` to detect the current runtime environment.
+Finally, the `kudu-test-utils` module provides the `KuduTestHarness` class,
+which runs a Kudu mini cluster.
+
+Maven example for Kudu 1.9.0:
+
+[source,xml]
+----
+<build>
+  <extensions>
+    <!-- Used to find the right kudu-binary artifact with the Maven
+         property ${os.detected.classifier} -->
+    <extension>
+      <groupId>kr.motd.maven</groupId>
+      <artifactId>os-maven-plugin</artifactId>
+      <version>1.6.2</version>
+    </extension>
+  </extensions>
+</build>
+
+<dependencies>
+  <dependency>
+    <groupId>org.apache.kudu</groupId>
+    <artifactId>kudu-test-utils</artifactId>
+    <version>1.9.0</version>
+    <scope>test</scope>
+  </dependency>
+  <dependency>
+    <groupId>org.apache.kudu</groupId>
+    <artifactId>kudu-binary</artifactId>
+    <version>1.9.0</version>
+    <classifier>${os.detected.classifier}</classifier>
+    <scope>test</scope>
+  </dependency>
+</dependencies>
+----
+
+**2. Write a test that starts a Kudu mini cluster using the KuduTestHarness.**
+It will automatically find the binary test JAR if Maven is configured correctly.
+
+The recommended way to start a Kudu mini cluster is by using the
+`KuduTestHarness` class from the `kudu-test-utils` module, which also acts as a
+JUnit `Rule`. Here is an example of a Java-based integration test that starts a
+Kudu cluster, creates a Kudu table on the cluster, and then exits:
+
+[source,java]
+----
+import org.apache.kudu.ColumnSchema;
+import org.apache.kudu.Schema;
+import org.apache.kudu.Type;
+import org.apache.kudu.test.KuduTestHarness;
+import org.junit.Rule;
+import org.junit.Test;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+public class MyKuduTest {
+  // The KuduTestHarness automatically starts and stops a real Kudu cluster
+  // when each test is run. Kudu persists its on-disk state in a temporary
+  // directory under a location defined by the environment variable TEST_TMPDIR
+  // if set, or under /tmp otherwise. That cluster data is deleted on
+  // successful exit of the test. The cluster output is logged through slf4j.
+  @Rule
+  public KuduTestHarness harness = new KuduTestHarness();
+
+  @Test
+  public void test() throws Exception {
+    // Get a KuduClient configured to talk to the running mini cluster.
+    KuduClient client = harness.getClient();
+
+    // Create a new Kudu table.
+    List<ColumnSchema> columns = new ArrayList<>();
+    columns.add(
+        new ColumnSchema.ColumnSchemaBuilder(
+            "key", Type.INT32).key(true).build());
+    Schema schema = new Schema(columns);
+    CreateTableOptions opts =
+        new CreateTableOptions().setRangePartitionColumns(
+            Collections.singletonList("key"));
+    client.createTable("table-1", schema, opts);
+
+    // Now we may insert rows into the newly-created Kudu table using 'client',
+    // scan the table, etc.
+  }
+}
+----
+
+For more examples of using the `KuduTestHarness`, including how to pass
+configuration options to the Kudu cluster being managed by the harness, see the
+link:https://github.com/apache/kudu/tree/master/examples/java/java-example[java-example]
+project in the Kudu source code repository, or look at the various Kudu
+integration tests under
+link:https://github.com/apache/kudu/tree/master/java[java] in the Kudu source
+code repository.
 
 == Kudu Python Client
 The Kudu Python client provides a Python friendly interface to the C++ client API.


Mime
View raw message