kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jtbirds...@apache.org
Subject [2/2] kudu git commit: [examples] KUDU-2382: Add Kudu Basic Python Example
Date Thu, 05 Jul 2018 17:48:53 GMT
[examples] KUDU-2382: Add Kudu Basic Python Example

This patch adds a basic Kudu Python Client example to the
code base. This example is also listed on the website.

Change-Id: I12016a7c9b1f4a1e20556297654c7abfa7b673cb
Reviewed-on: http://gerrit.cloudera.org:8080/10794
Tested-by: Kudu Jenkins
Reviewed-by: Will Berkeley <wdberkeley@gmail.com>


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/c5df00c8
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/c5df00c8
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/c5df00c8

Branch: refs/heads/master
Commit: c5df00c8f5a63115d0545f254605658676f3bc88
Parents: a1558ee
Author: Jordan Birdsell <jtbirdsell@apache.org>
Authored: Fri Jun 22 09:55:14 2018 -0500
Committer: Jordan Birdsell <jtbirdsell@apache.org>
Committed: Thu Jul 5 17:48:10 2018 +0000

----------------------------------------------------------------------
 examples/python/basic-python-example/README.md  | 51 ++++++++++++
 .../basic-python-example/basic_example.py       | 88 ++++++++++++++++++++
 2 files changed, 139 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/c5df00c8/examples/python/basic-python-example/README.md
----------------------------------------------------------------------
diff --git a/examples/python/basic-python-example/README.md b/examples/python/basic-python-example/README.md
new file mode 100644
index 0000000..ab62de3
--- /dev/null
+++ b/examples/python/basic-python-example/README.md
@@ -0,0 +1,51 @@
+<!---
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Basic Kudu-Python Example
+This is a very basic example of usage for the Kudu Python client.
+It demonstrates much of the standard capabilities within the client.
+
+## To install the Python client
+
+### Building from source
+_NOTE:_ This example is pointing to the latest build, which is often
+times a debug build. If this is not the desired approach, you will
+need to point this to the release build.
+```
+export KUDU_HOME=/path/to/kudu
+export LD_LIBRARY_PATH=$KUDU_HOME/build/latest/lib/exported
+# For OS X
+export DYLD_LIBRARY_PATH=$KUDU_HOME/build/latest/lib/exported
+pip install -r requirements.txt
+python setup.py build_ext --inplace
+python setup.py install
+```
+
+### Installing from pypi
+_NOTE:_ This example is pointing to the latest build, which is often
+times a debug build. If this is not the desired approach, you will
+need to point this to the release build.
+```
+export KUDU_HOME=/path/to/kudu
+export LD_LIBRARY_PATH=$KUDU_HOME/build/latest/lib/exported
+# For OS X
+export DYLD_LIBRARY_PATH=$KUDU_HOME/build/latest/lib/exported
+pip install kudu-python
+```
+
+## Running the example
+
+```
+./basic_example.py --masters master1.address --ports 7051
+```
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/kudu/blob/c5df00c8/examples/python/basic-python-example/basic_example.py
----------------------------------------------------------------------
diff --git a/examples/python/basic-python-example/basic_example.py b/examples/python/basic-python-example/basic_example.py
new file mode 100644
index 0000000..e0557a3
--- /dev/null
+++ b/examples/python/basic-python-example/basic_example.py
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime
+import argparse
+
+import kudu
+from kudu.client import Partitioning
+
+
+# Parse arguments
+parser = argparse.ArgumentParser(description='Basic Example for Kudu Python.')
+parser.add_argument('--masters', '-m', nargs='+', default='localhost',
+                    help='The master address(es) to connect to Kudu.')
+parser.add_argument('--ports', '-p', nargs='+', default='7051',
+                    help='The master server port(s) to connect to Kudu.')
+args = parser.parse_args()
+
+
+# Connect to Kudu master server(s).
+client = kudu.connect(host=args.masters, port=args.ports)
+
+# Define a schema for a new table.
+builder = kudu.schema_builder()
+builder.add_column('key').type(kudu.int64).nullable(False).primary_key()
+builder.add_column('ts_val', type_=kudu.unixtime_micros, nullable=False, compression='lz4')
+schema = builder.build()
+
+# Define the partitioning schema.
+partitioning = Partitioning().add_hash_partitions(column_names=['key'], num_buckets=3)
+
+# Delete table if it already exists.
+if client.table_exists('python-example'):
+  client.delete_table('python-example')
+
+# Create a new table.
+client.create_table('python-example', schema, partitioning)
+
+# Open a table.
+table = client.table('python-example')
+
+# Create a new session so that we can apply write operations.
+session = client.new_session()
+
+# Insert a row.
+op = table.new_insert({'key': 1, 'ts_val': datetime.utcnow()})
+session.apply(op)
+
+# Upsert a row.
+op = table.new_upsert({'key': 2, 'ts_val': "2016-01-01T00:00:00.000000"})
+session.apply(op)
+
+# Update a row.
+op = table.new_update({'key': 1, 'ts_val': ("2017-01-01", "%Y-%m-%d")})
+session.apply(op)
+
+# Delete a row.
+op = table.new_delete({'key': 2})
+session.apply(op)
+
+# Flush write operations, if failures occur, print them.
+try:
+  session.flush()
+except kudu.KuduBadStatus:
+  print(session.get_pending_errors())
+
+# Create a scanner and add a predicate.
+scanner = table.scanner()
+scanner.add_predicate(table['ts_val'] == datetime(2017, 1, 1))
+
+# Open scanner and print all tuples.
+# Note: This doesn't scale for large scans
+# Output: [(1, datetime.datetime(2017, 1, 1, 0, 0, tzinfo=<UTC>))]
+print(scanner.open().read_all_tuples())


Mime
View raw message