kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "helifu" <hzhel...@corp.netease.com>
Subject 答复: Tablets Per Tablet Server
Date Fri, 30 Nov 2018 01:03:10 GMT
Hi Mac,

 

To many of the small tables which have the minimum of 2 partitions, there is another way to
help you to save half the tablets.(Thanks to Andrew, who shared this info to me on the slack)

You can find the detail in both link: 

https://jira.apache.org/jira/browse/IMPALA-5546   IMPALA-2.10.0

https://www.cloudera.com/documentation/enterprise/latest/topics/kudu_impala.html#concept_g51_5vk_ft

 

Note: In Impala included in CDH 5.13 and higher, the PARTITION BY clause is optional for Kudu
tables. If the clause is omitted, Impala automatically constructs a single partition that
is not connected to any column. Because such a table cannot take advantage of Kudu features
for parallelized queries and query optimizations, omitting the PARTITION BY clause is only
appropriate for small lookup tables.

 

 

何李夫

2018-11-30 09:02:21

 

发件人: user-return-1546-hzhelifu=corp.netease.com@kudu.apache.org <user-return-1546-hzhelifu=corp.netease.com@kudu.apache.org>
代表 Mac Noland
发送时间: 2018年11月30日 1:58
收件人: user@kudu.apache.org
主题: Tablets Per Tablet Server

 

Good day,

 

We have a number of customers using Kudu for a large number of tables from db/warehouse offload
use cases and commonly run into situations where our tablets are over 2000 per tablet server.
 Many of the tables are actually small, but with the minimum of 2 partitions and default of
3 replicas, we end up chewing up 6 tablets minimally for each table.  Being we have customers
way over this and are running fine, we understand this is not a hard guideline, but overall
does cause some concern with some customers knowing we are outside of the written guidelines.

 

The maximum number of tablets per tablet server is 2000, post-replication, but we recommend
1000 tablets or fewer per tablet server.  https://kudu.apache.org/docs/known_issues.html#_scale

 

Do we know how this number was picked and if there is any work being done to look at what
it would take to increase the number?  Or things we could contribute to help increase that
number?  Not 100% sure what we'd be signing up for to do that, but always looking for opportunity
to help.

 

Thanks in advance.

 

Mac

 

 


Mime
View raw message