kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From granthe...@apache.org
Subject [kudu] 02/02: release notes: add prior notes for 1.10.0
Date Tue, 09 Jul 2019 17:03:30 GMT
This is an automated email from the ASF dual-hosted git repository.

granthenke pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit 43f112ec3e9c2285988fb56abe09fe8e3287eecf
Author: Grant Henke <granthenke@apache.org>
AuthorDate: Tue Jul 9 10:41:54 2019 -0500

    release notes: add prior notes for 1.10.0
    Change-Id: If783f945d7b739dd2df24f58d43b4a0c89ac3e71
    Reviewed-on: http://gerrit.cloudera.org:8080/13825
    Tested-by: Kudu Jenkins
    Reviewed-by: Andrew Wong <awong@cloudera.com>
 docs/prior_release_notes.adoc | 281 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 281 insertions(+)

diff --git a/docs/prior_release_notes.adoc b/docs/prior_release_notes.adoc
index 74b5011..7818953 100644
--- a/docs/prior_release_notes.adoc
+++ b/docs/prior_release_notes.adoc
@@ -37,6 +37,287 @@ reproduced on this page. Please consult the
 link:http://kudu.apache.org/releases/[documentation of the appropriate release]
 for a list of known issues and limitations.
+== Release Notes Specific to 1.10.0
+== Upgrade Notes
+* The default tablet history retention time has been raised from 15 minutes to
+  7 days to better support touchless incremental backups
+  (see link:https://issues.apache.org/jira/browse/KUDU-2677[KUDU-2677]).
+== Obsoletions
+== Deprecations
+== New features
+* Kudu now supports both full and incremental table backups via a job
+  implemented using Apache Spark. Additionally it supports restoring
+  tables from full and incremental backups via a restore job implemented using
+  Apache Spark. See the link:administration.html#backup[backup documentation]
+  for more details.
+* Kudu can now synchronize its internal catalog with the Apache Hive Metastore,
+  automatically updating Hive Metastore table entries upon table creation,
+  deletion, and alterations in Kudu. See the
+  link:hive_metastore.html#metadata_sync[HMS synchronization documentation]
+  for more details.
+* Kudu now supports native fine-grained authorization via integration with
+  Apache Sentry. Kudu may now enforce access control policies defined for Kudu
+  tables and columns, as well as policies defined on Hive servers and databases
+  that may store Kudu tables. See the
+  link:security.html#fine_grained_authz[authorization documentation] for more
+  details.
+* Kudu’s web UI now supports SPNEGO, a protocol for securing HTTP requests with
+  Kerberos by passing negotiation through HTTP headers. To enable, set the
+  `--webserver_require_spnego` command line flag.
+* Column comments can now be stored in Kudu tables, and can be updated using
+  the AlterTable API
+  (see link:https://issues.apache.org/jira/browse/KUDU-1711[KUDU-1711]).
+* The Java scan token builder can now create multiple tokens per tablet.
+  To use this functionality, call `setSplitSizeBytes()` to specify how many bytes
+  of data each token should scan. The same API is also available in Kudu’s
+  Spark integration, where it can be used to spawn multiple Spark tasks per
+  scanned tablet
+  (see link:https://issues.apache.org/jira/browse/KUDU-2670[KUDU-2670]).
+* Experimental Kudu Docker images are now published on
+  link:https://hub.docker.com/r/apache/kudu[Docker Hub].
+* Kudu now has an experimental Kubernetes StatefulSet manifest and Helm chart,
+  which can be used to define and provision Kudu clusters using Kubernetes
+  (see link:https://issues.apache.org/jira/browse/KUDU-2398[KUDU-2398]).
+* The Kudu CLI now has rudimentary YAML-based configuration file support,
+  which can be used to provide cluster connection information via cluster name
+  instead of keying in comma-separated lists of master addresses.
+  See the link:https://kudu.apache.org/docs/administration.html#using_cluster_names_in_kudu_tool[cluster
name documentation]
+  for more details.
+* `kudu perf table_scan` scans a table and displays a table’s row count as well
+  as the time it took to run the scan.
+* `kudu table copy` copies data from one table to another, within the same
+  cluster or across clusters. Note, this implementation leverages a single client,
+  therefore it may not be suitable for large tables.
+* Tablet history retention time can now be configured on a table-by-table basis.
+  (see link:https://issues.apache.org/jira/browse/KUDU-2514[KUDU-2514]).
+== Optimizations and improvements
+* The performance of mutations (i.e. UPDATE, DELETE, and re-INSERT) to
+  not-yet-flushed Kudu data has been significantly optimized
+  (see link:https://issues.apache.org/jira/browse/KUDU-2826[KUDU-2826] and
+  link:https://github.com/apache/kudu/commit/f9f9526d3[f9f9526d3]).
+* Predicate performance for primitive columns has been optimized
+  (see link:https://issues.apache.org/jira/browse/KUDU-2846[KUDU-2846]).
+* IS NULL and IS NOT NULL predicate performance has been optimized
+  (see link:https://issues.apache.org/jira/browse/KUDU-2846[KUDU-2846]).
+* Optimized the performance of fetching tablet locations from the master for
+  tables with large numbers of partitions. This can improve the performance of
+  short-running Spark or Impala queries as well as user applications which make
+  use of short-lived client instances
+  (see link:https://issues.apache.org/jira/browse/KUDU-2711[KUDU-2711]).
+* The `tableExists()` (Java) and `TableExists()` (C++) APIs are now more performant
+  (see link:https://issues.apache.org/jira/browse/KUDU-2802[KUDU-2802]).
+* Fault tolerant scans are now much more performant and consume far less memory
+  (see link:https://issues.apache.org/jira/browse/KUDU-2466[KUDU-2466]).
+* `kudu cluster ksck` now sends more requests in parallel, which should result
+  in a speed-up when running against clusters with many tables or when there’s
+  high latency between the node running the CLI and the cluster nodes.
+* Kudu’s block manager now deletes spent block containers when needed instead of
+  just at server startup. This should reduce server startup times somewhat
+  (see link:https://issues.apache.org/jira/browse/KUDU-2636[KUDU-2636]).
+* DNS resolutions are now cached by Kudu masters, tablet servers, and
+  Kudu C++ clients. The TTL for a resolved DNS entry in the cache is 15 seconds
+  by default
+  (see link:https://issues.apache.org/jira/browse/KUDU-2791[KUDU-2791]).
+* Tables created in Kudu 1.10.0 or later will show their creation time as well
+  as their last alteration time in the web UI
+  (see link:https://issues.apache.org/jira/browse/KUDU-2750[KUDU-2750]).
+* The Kudu CLI and C++ client now support overriding the local username using
+  the ‘KUDU_USER_NAME’ environment variable. This allows operating against a
+  Kudu cluster using an identity which differs from the local Unix user on the
+  client. Note that this has no effect on secure clusters, where client identity
+  is determined by Kerberos authentication
+  (see link:https://issues.apache.org/jira/browse/KUDU-2717[KUDU-2717]).
+* Kudu C++ client now performs stricter verification on the input data of
+  INSERT and UPSERT operations w.r.t. table schema constraints. This helps
+  spotting schema violations before sending the data to a tablet server.
+* The `KuduScanner` in the Java client is now iterable. Additionally the
+  `KuduScannerIterator` will automatically make scanner keep alive calls to
+  ensure scanners do not time out while iterating.
+* A `KuduPartitioner` API was added to the Java client. The `KuduPartitioner`
+  API allows a client to determine which partition a row falls into without
+  actually writing that row. For example, the `KuduPartitioner` is used in the
+  Spark integration to optionally repartition and pre-sort the data before
+  writing to Kudu
+  (see link:https://issues.apache.org/jira/browse/KUDU-2674[KUDU-2674] and
+  link:https://issues.apache.org/jira/browse/KUDU-2672[KUDU-2672]).
+* The `PartialRow` and `RowResult` Java API have new methods that accept and return
+  Java Objects. These methods are useful when you don't care about autoboxing
+  and your existing type handling logic is based on Java types. See the javadoc
+  for more details.
+* The Kudu Java client now logs RPC trace summaries instead of full RPC traces when
+  the log level is `INFO` or higher. This reduces log noise and makes RPC issues
+  more visible in a more compact format
+  (see link:https://issues.apache.org/jira/browse/KUDU-2830[KUDU-2830]).
+* Kudu servers now display the time at which they were started in their web UIs.
+* Kudu tablet servers now display a table’s total column count in the web UI.
+* The `/metrics` web UI endpoint now supports filtering on entity types,
+  entity IDs, entity attributes, and metric names. This can be used to more
+  efficiently collect important metrics when there is a large number of tablets
+  on a tablet server.
+* The Kudu rebalancer now accepts the `--ignored_tservers` command line
+  argument, which can be used to ignore the health status of specific tablet
+  servers (i.e. if they are down) when deciding whether or not it’s safe to
+  rebalance the cluster.
+* `kudu master list` now displays the Raft consensus role of each master in the
+  cluster (i.e. LEADER or FOLLOWER)
+  (see link:https://issues.apache.org/jira/browse/KUDU-2825[KUDU-2825]).
+* `kudu table scan` no longer interleaves its output, and now projects all
+  columns without having to manually list the column names.
+* `kudu perf loadgen` now supports creating empty tables. The semantics of the
+  special value of 0 for `--num_rows_per_thread` flag has changed. A value of 0
+  now indicates that no rows should be generated, and a value of -1 indicates
+  there should be no limit to the number of rows generated.
+* Running `make install` after building Kudu from source will now install the
+  Kudu binaries into appropriate locations.
+  (see link:https://issues.apache.org/jira/browse/KUDU-1344[KUDU-1344]).
+== Fixed Issues
+* Fixed an issue where the Java client would fail scans that took a very long
+  time to return a single block of rows, such as highly selective scans over a
+  large amount of data
+  (see link:https://issues.apache.org/jira/browse/KUDU-1868[KUDU-1868]).
+* Fixed the handling of SERVICE_UNAVAILABLE errors that caused the Java client
+  to do unnecessary master lookups.
+* Kudu scan tokens now work correctly when the target table is renamed between
+  when the scan token is created and when it is rehydrated into a scanner.
+* Kudu’s “NTP synchronization wait” behavior at startup now works properly when
+  Kudu is run in a containerized environment.
+* Fixed a crash when a flush or compaction overlapped with another compaction
+  (see link:https://issues.apache.org/jira/browse/KUDU-2807[KUDU-2807]).
+* Fixed a rare race at startup where the leader master would fruitlessly try to
+  tablet copy to a healthy follower master, causing the cluster to operate as if
+  it had two masters until master leadership changed
+  (see link:https://issues.apache.org/jira/browse/KUDU-2748[KUDU-2748]).
+* Under rare circumstances, it was possible for Kudu to crash in libkrb5 when
+  negotiating multiple TLS connections concurrently. This crash has been fixed
+  (see link:https://issues.apache.org/jira/browse/KUDU-2706[KUDU-2706]).
+* Kudu no longer crashes at startup on machines with disabled CPUs
+  (see link:https://issues.apache.org/jira/browse/KUDU-2721[KUDU-2721]).
+== Wire Protocol compatibility
+Kudu 1.10.0 is wire-compatible with previous versions of Kudu:
+* Kudu 1.10 clients may connect to servers running Kudu 1.0 or later. If the client uses
+  features that are not available on the target server, an error will be returned.
+* Rolling upgrade between Kudu 1.9 and Kudu 1.10 servers is believed to be possible
+  though has not been sufficiently tested. Users are encouraged to shut down all nodes
+  in the cluster, upgrade the software, and then restart the daemons on the new version.
+* Kudu 1.0 clients may connect to servers running Kudu 1.10 with the exception of the
+  below-mentioned restrictions regarding secure clusters.
+The authentication features introduced in Kudu 1.3 place the following limitations
+on wire compatibility between Kudu 1.10 and versions earlier than 1.3:
+* If a Kudu 1.10 cluster is configured with authentication or encryption set to "required",
+  clients older than Kudu 1.3 will be unable to connect.
+* If a Kudu 1.10 cluster is configured with authentication and encryption set to "optional"
+  or "disabled", older clients will still be able to connect.
+== Incompatible Changes in Kudu 1.10.0
+* Support for building and running with Java 7 has been dropped in this release.
+  It had been deprecated since Kudu 1.5.0.
+  (see link:https://issues.apache.org/jira/browse/KUDU-2099[KUDU-2099]).
+=== Client Library Compatibility
+* The Kudu 1.10 Java client library is API- and ABI-compatible with Kudu 1.9. Applications
+  written against Kudu 1.9 will compile and run against the Kudu 1.10 client library and
+  vice-versa.
+* The Kudu 1.10 {cpp} client is API- and ABI-forward-compatible with Kudu 1.9.
+  Applications written and compiled against the Kudu 1.9 client library will run without
+  modification against the Kudu 1.10 client library. Applications written and compiled
+  against the Kudu 1.10 client library will run without modification against the Kudu 1.9
+  client library.
+* The Kudu 1.10 Python client is API-compatible with Kudu 1.9. Applications
+  written against Kudu 1.9 will continue to run against the Kudu 1.10 client
+  and vice-versa.
+== Known Issues and Limitations
+Please refer to the link:known_issues.html[Known Issues and Limitations] section of the
+== Contributors
+Kudu 1.10 includes contributions from 27 people, including 6 first-time contributors:
+* Csaba Fulop
+* Florentino Sainz
+* Guangchao Deng
+* Jia Hongchao
+* Ye Yuqiang
+* Yifan Zhang
+Thank you for your help in making Kudu even better!
 == Release Notes Specific to 1.9.0

View raw message