From user-return-1498-archive-asf-public=cust-asf.ponee.io@kudu.apache.org Wed Oct 10 23:04:57 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1EE41180672 for ; Wed, 10 Oct 2018 23:04:56 +0200 (CEST) Received: (qmail 42132 invoked by uid 500); 10 Oct 2018 21:04:56 -0000 Mailing-List: contact user-help@kudu.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@kudu.apache.org Delivered-To: mailing list user@kudu.apache.org Received: (qmail 42121 invoked by uid 99); 10 Oct 2018 21:04:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Oct 2018 21:04:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 137021A1B1A for ; Wed, 10 Oct 2018 21:04:55 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.2 X-Spam-Level: * X-Spam-Status: No, score=1.2 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=boristyukin.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id 007rsVrBwNS4 for ; Wed, 10 Oct 2018 21:04:50 +0000 (UTC) Received: from mx36-out26.antispamcloud.com (mx36-out26.antispamcloud.com [209.126.121.74]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id A67A05F35B for ; Wed, 10 Oct 2018 21:04:49 +0000 (UTC) Received: from s2.fcomet.com ([99.198.101.250]) by mx61.antispamcloud.com with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1gALeQ-000CVW-7L for user@kudu.apache.org; Wed, 10 Oct 2018 23:04:40 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=boristyukin.com; s=default; h=Content-Type:To:Subject:Message-ID:Date:From: MIME-Version:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Po1RcriitGmJVlOnxFHOOiWAEfRIuM2I+9FptKHmoxM=; b=wwnxMVcceUcj3gLoXDSwFLzSiZ 1ljzcaSHt1CFiHJwPb2iOp0HNLSQYs0vaZqlEtHVHM5WaYIr11kBTCszsz+X9YSmLjINhC0nLh3yd fyjMNgvw3weL6W1Nugiy8AUYS8QaRv/P3yY79UfrlDYRLq5zw9sTo5elI0yBomG/A3pY=; Received: from mail-it1-f176.google.com ([209.85.166.176]:56189) by s2.fcomet.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from ) id 1gALdi-001nJg-Jt for user@kudu.apache.org; Wed, 10 Oct 2018 16:03:30 -0500 Received: by mail-it1-f176.google.com with SMTP id c23-v6so10244538itd.5 for ; Wed, 10 Oct 2018 14:03:31 -0700 (PDT) X-Gm-Message-State: ABuFfojfC5lrVPYdJdXm/RvfM2R7FPhzFIro8pX65MBWNxAF+DVIiPDP S92gteqvGaKZV02ntGqmjuh5OyjqC5dx5GmDXdY= X-Google-Smtp-Source: ACcGV62A1X3eLbcftC5K49rCo/qah15UcyA9eF5DPaIofg9iWQ8Vz8htN09AENo8yLfrT+wj1zwDAPqBPKmG+NN/G/s= X-Received: by 2002:a02:94cc:: with SMTP id x70-v6mr28032348jah.74.1539205410749; Wed, 10 Oct 2018 14:03:30 -0700 (PDT) MIME-Version: 1.0 From: Boris Tyukin Date: Wed, 10 Oct 2018 17:02:54 -0400 X-Gmail-Original-Message-ID: Message-ID: Subject: clarification on Partitioning Guidelines and CPU cores To: user@kudu.apache.org Content-Type: multipart/alternative; boundary="000000000000be8e030577e62feb" X-AuthUser: boris@boristyukin.com X-Originating-IP: 99.198.101.250 X-AntiSpamCloud-Domain: s2.fcomet.com X-AntiSpamCloud-Username: 99.198.101.250 Authentication-Results: antispamcloud.com; auth=pass smtp.auth=99.198.101.250@s2.fcomet.com X-AntiSpamCloud-Outgoing-Class: unsure X-AntiSpamCloud-Outgoing-Evidence: Combined (0.22) X-Recommended-Action: accept X-Filter-ID: EX5BVjFpneJeBchSMxfU5kuFDdX7diqiYelS7wdmHbR602E9L7XzfQH6nu9C/Fh9KJzpNe6xgvOx q3u0UDjvO2HFEYQDlNqPthodLGs7Ym7sT+bYMn6GaM87VISxooGyDkRcwtIq0ugR+f026HFut1pr jQPFk8m4tSTfORUp3ynEm+h0A2koB3qKN5bbUQlCvB1aHZYTYX2JaePLXtK5ho32ycKuYR+eCujo ZgTPFnZKenuD+fJRvZgsOGa/86DNKY4i2I4KjPTYrQ+5jVlmW+/cSY+GknWNdoEa0JVhYAcTmEFB zGJ4I3iI+cUBLpxHZqMsFXHkY4b0tMjYHlbEsQ6yCIj+j+sW7DnHSTh9wIB9qzbFFetd0V4Svjqc FPolF8VpymDAm51vSeaku+X2qYfWdHzoB2yW92yX6zvRr8FsMyewd83qhRHUN79OaH7QZM3NuHht 4ah3jKpVe9Q5dQwPf1ekvTsJu+noz87G5EthgB37cSRdX+S8eSGMVvuOehIqUczFWeS6sE8e1b5/ UuG84yBZdFDsZzjqLz0+lM3AQMMUBb2t50QJMtcbJcmmbRL4Heo9b9yf4iFRBMlkSAglzDSkmu+2 ngWbLO8BK3gHmFDqewO9xyOqCYO8P1aHtoY6rzqyKqE9hWj/+i04wxL4LGvpf1LjyWJ3NV8yWSND ebtqDywdo+KNR3RjhF85voJqRaAXkPdTBzDG5RxHbE8cW9l7L/bUyy3TdA61l2dlx/K20P76Qx6z cTdj26a2r5JrWjOM89ubFiKM88lx8MTTuJLgczuNevh7KO3fJ05T1Eumqg7wtc13Hpo3rogN9R/2 gMGq0KWAzmMf+ibVDiVWkOP3Qp/rLANvEsrAznpYdwcbZJUcRJkmNgMr2kgJ0H3fzGZNZsYwNt3p c7yxqwPDd1NsXyfFHjI8dxZkd/IDWDyg/Q/09Ieu7aO1pJGxcKFSaLVuBqcLhjlJqmNWEfY4ocfm Wv3Fe9Iziczdq+A= X-Report-Abuse-To: spam@quarantine1.antispamcloud.com --000000000000be8e030577e62feb Content-Type: text/plain; charset="UTF-8" Hi all, can someone clarify if this recommendation below - does it mean physical or hyper-threaded CPU cores? quite a big difference... Thanks, Boris Partitioning Guidelines (https://kudu.apache.org/docs/ kudu_impala_integration.html#partitioning_rules_of_thumb) - For large tables, such as fact tables, aim for as many tablets as you have cores in the cluster. - For small tables, such as dimension tables, aim for a large enough number of tablets that each tablet is at least 1 GB in size. In general, be mindful the number of tablets limits the parallelism of reads, in the current implementation. Increasing the number of tablets significantly beyond the number of cores is likely to have diminishing returns. --000000000000be8e030577e62feb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi all,

can someone clarify if this rec= ommendation below - does it mean physical or hyper-threaded CPU cores? quit= e a big difference...
Thanks,
Boris

Partitioning Guidelines (https:<= span style=3D"color:rgb(119,119,119)">//kudu.apache.org= /docs/kudu_impala_int= egration.html#partitioni= ng_rules_of_thumb)
- For large tables, such as = fact tables, aim for as many t= ablets as you have cores in the cluster.
- For small tables, such as dimension tables, aim for a large enou= gh number of tablets that each= tablet is at least 1 GB in size.

In general, be mindful the <= span style=3D"color:rgb(122,62,157)">number of tablets limits the pa= rallelism of reads, in the cur= rent implementation. Increasing the n= umber of tablets significantly beyond the number of cores is= likely to have diminis= hing returns.

--000000000000be8e030577e62feb--