kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From granthe...@apache.org
Subject [kudu] 01/02: [docs] Update the limitations page
Date Wed, 03 Jul 2019 02:03:19 GMT
This is an automated email from the ASF dual-hosted git repository.

granthenke pushed a commit to branch branch-1.10.x
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit fdabd45eb78ddf67f8c676e2228f25b4a6c863cc
Author: Grant Henke <granthenke@apache.org>
AuthorDate: Thu Jun 27 20:19:16 2019 -0500

    [docs] Update the limitations page
    
    This patch updates the scaling limitations to be a bit more
    nuanced.  We have gotten feedback that users think the
    limits listed are hard and fast rules and this change is
    intended to be more clear about “just works” scale
    vs some of the largest scales/configurations that have
    been seen/reported.
    
    A community survey will be conducted soon to adjust
    these values further.
    
    I also adjusted a few other out of date limitations.
    
    Change-Id: Iafa4f7f3bd9e405f7ffc4e6cde48ec28e6e04081
    Reviewed-on: http://gerrit.cloudera.org:8080/13756
    Reviewed-by: Andrew Wong <awong@cloudera.com>
    Tested-by: Grant Henke <granthenke@apache.org>
    (cherry picked from commit 2af0983941f79daae1964972b66dc736ee8d8713)
    Reviewed-on: http://gerrit.cloudera.org:8080/13791
---
 docs/known_issues.adoc | 57 ++++++++++++++++++++++++++++++--------------------
 1 file changed, 34 insertions(+), 23 deletions(-)

diff --git a/docs/known_issues.adoc b/docs/known_issues.adoc
index 3eddd8c..2c9d9bf 100644
--- a/docs/known_issues.adoc
+++ b/docs/known_issues.adoc
@@ -51,13 +51,13 @@
 
 === Columns
 
-* CHAR, VARCHAR, DATE, and complex types like ARRAY are not supported.
+* CHAR, VARCHAR, DATE, and complex types like ARRAY, MAP, and STRUCT are not supported.
 
 * Type and nullability of existing columns cannot be changed by altering the table.
 
 * The precision and scale of `DECIMAL` columns cannot be changed by altering the table.
 
-* Tables can have a maximum of 300 columns.
+* Tables can have a maximum of 300 columns by default.
 
 === Tables
 
@@ -132,33 +132,43 @@
 
 == Scale
 
-* Recommended maximum number of tablet servers is 100.
+Kudu is known to run seamlessly across a wide array of environments and workloads
+with minimal expertise and configuration at the following scale:
 
-* Recommended maximum number of masters is 3.
+* 3 master servers
 
-* Recommended maximum amount of stored data, post-replication and post-compression,
-  per tablet server is 8 TiB.
+* 100 tablet servers
 
-* The maximum number of tablets per tablet server is 2000, post-replication,
-  but we recommend 1000 tablets or fewer per tablet server.
+* 8 TiB of stored data per tablet server, post-replication and post-compression.
 
-* Maximum number of tablets per table for each tablet server is 60,
-  post-replication (assuming the default replication factor of 3), at table-creation time.
+* 1000 tablets per tablet server, post-replication.
 
-* Recommended maximum amount of data per tablet is 50 GiB. Going beyond this can cause
-  issues such a reduced performance, compaction issues, and slow tablet startup times.
-  The recommended target size for tablets is under 10 GiB.
+* 60 tablets per table, per tablet server, at table-creation time.
 
-== Replication and Backup Limitations
+* 10 GiB of stored data per tablet.
 
-* Kudu does not currently include any built-in features for backup and restore.
-  Users are encouraged to use tools such as Spark or Impala to export or import
-  tables as necessary.
+Staying within these limits will provide the most predictable and straightforward
+Kudu experience.
+
+However, experienced users who run on modern hardware, use the latest
+versions of Kudu, test and tune Kudu for their use case, and work closely with
+the community, can achieve much higher scales comfortably. Below are some
+anecdotal values that have been seen in real world production clusters:
+
+* 3 master servers
+
+* 300+ tablet servers
+
+* 10+ TiB of stored data per tablet server, post-replication and post-compression.
+
+* 4000+ tablets per tablet server, post-replication.
+
+* 50 GiB of stored data per tablet. Going beyond this can cause issues such a
+  reduced performance, compaction issues, and slow tablet startup times.
 
 == Security Limitations
 
-* Authorization is only available at a system-wide, coarse-grained level. Table-level,
-  column-level, and row-level authorization features are not available.
+* Row-level authorization is not available.
 
 * Data encryption at rest is not directly built into Kudu. Encryption of
   Kudu data at rest can be achieved through the use of local block device
@@ -169,7 +179,9 @@
 
 * Server certificates generated by Kudu IPKI are incompatible with
   link:https://www.bouncycastle.org/[bouncycastle] version 1.52 and earlier. See
-  link:https://issues.apache.org/jira/browse/KUDU-2145[KUDU-2145] for details.
+  link:https://issues.apache.org/jira/browse/KUDU-2145[KUDU-2145] for details.'
+
+* The highest supported version of the TLS protocol is TLSv1.2
 
 == Other Known Issues
 
@@ -182,6 +194,5 @@ to communicate only the most important known issues.
 
 * If a tablet server has a very large number of tablets, it may take several minutes
   to start up. It is recommended to limit the number of tablets per server to 1000
-  or fewer. The maximum allowed number of tablets per server is 2000.
-  Consider this limitation when pre-splitting your tables. If you notice slow start-up times,
-  you can monitor the number of tablets per server in the web UI.
+  or fewer. Consider this limitation when pre-splitting your tables. If you notice slow
+  start-up times, you can monitor the number of tablets per server in the web UI.


Mime
View raw message