From user-return-1484-archive-asf-public=cust-asf.ponee.io@kudu.apache.org Tue Sep 4 19:12:13 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id D945A180629 for ; Tue, 4 Sep 2018 19:12:12 +0200 (CEST) Received: (qmail 58051 invoked by uid 500); 4 Sep 2018 17:12:07 -0000 Mailing-List: contact user-help@kudu.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@kudu.apache.org Delivered-To: mailing list user@kudu.apache.org Received: (qmail 58040 invoked by uid 99); 4 Sep 2018 17:12:06 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Sep 2018 17:12:06 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id EA23EC1DEE for ; Tue, 4 Sep 2018 17:12:05 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=cloudera.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 3xrPRq3L8V-n for ; Tue, 4 Sep 2018 17:12:03 +0000 (UTC) Received: from mail-oi0-f46.google.com (mail-oi0-f46.google.com [209.85.218.46]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id A4D795F3EC for ; Tue, 4 Sep 2018 17:12:02 +0000 (UTC) Received: by mail-oi0-f46.google.com with SMTP id x197-v6so8168651oix.5 for ; Tue, 04 Sep 2018 10:12:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudera.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=axGLcgnQ+HIBuu3iSzH2XuSlpLvMpkVwU1PMjHHTvlw=; b=QdEervMEYRqXGGbdEsiBv387MS4d4LNpAhbKTKMGaDi7P4XNj8zzKH81i+M3+EB0JA S2JtatJnm2LwYK8zS0n9JV+lVzNq78gIdHC5gr6Ywn5aaffptksi4I5jka0WIJKjae4K HtCZF2NxVvxFnJ3b6yBgD57hSOJH5t60z+Q6ApFIeZOPp0jekbBqT9fYWh1BM+Am3wAx UmajOIZnWBAK9wJXArNIuX6eMlD4TMEhbGaw35iagGqdY8rhtXg8j2ktzeMzmfum69lQ 8l8R0qzMJdYNt5MfRI7fTrqlolwkXRufqwrlyEndHNoSBAcl9yrUsBC3JR2qKh1TRfTC UYdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=axGLcgnQ+HIBuu3iSzH2XuSlpLvMpkVwU1PMjHHTvlw=; b=pL9G3eGopraLnN/TVUSoW5p2sqwDv2EmnusSCpXPKw6rDaXWWcbQUCP+R9QnwvLN5Z o0Bzxc8kHMpUFLM3v1DjBaCSGOBAE1smxHZIwIfgryZUkqDdHKRwsM2P8TcutqRalpTO pRuvi6UV8TxkXlb9Oxh9zt+LLgkw1uaQVv3A0a0ZJeZsyNh8mUBbGzO1CmBEPQ4wfia4 y6DMKZ4c9VO3X6oAyh7eYR8jPDbSZpJ1SqsRSdGCpl5v5IKUhjcBgEKTHBKCmAhpyGuC zCprRitU2GSmM8vSXXX7MhnpYfT3ifIsGedsZMQPSQTNcEk0EbJ/Z+dQA3vhK757ywDO bb2Q== X-Gm-Message-State: APzg51Ay5mLAMD3gm2WycUmaKQJRYfRg/Es2kz77gtM/CQJfkiKOJEGj lHJwN1sbtN1EzcKH84I2R1cFM+M+PNKNirwauppv13UM X-Google-Smtp-Source: ANB0VdauUeXsmkwppxvaJCGSFkB8MtDunAPBHyjd3gcWFrj3EZDj1U9SxmK2RrlQWsisYqVsfobqnTcu6BZbwyL9x1s= X-Received: by 2002:aca:bbc4:: with SMTP id l187-v6mr23547059oif.278.1536081120277; Tue, 04 Sep 2018 10:12:00 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: William Berkeley Date: Tue, 4 Sep 2018 10:11:49 -0700 Message-ID: Subject: Re: Kudu's data pagination To: user@kudu.apache.org Content-Type: multipart/alternative; boundary="000000000000855b1105750ec12f" --000000000000855b1105750ec12f Content-Type: text/plain; charset="UTF-8" Hi Irtiza. What do you mean by paginate? I'm guessing you mean doing something like taking the results of a query like SELECT name, age FROM users SORT BY age DESC and displaying the results on some UI 10 at a time, say. If that's the case, the answer is no. It requires additional application code. In general, Kudu cannot return rows in order. So, if you want rows 101-110, you must retrieve *all* the rows, select the top 110, and then display only the final 10. In special cases when the sort is on a prefix of the primary key, scan tokens can be used to have Kudu return sorted subsets of rows from each tablet, which you can partially merge to get the desired result set. With a lot of data it's best to retrieve a large amount of sorted results and paginate from the cached results, rather than running a new query per page. -Will On Tue, Sep 4, 2018 at 9:02 AM Irtiza Ali wrote: > Hello everyone, > > Is there a way to paginate kudu's data using its python client? > > > I > --000000000000855b1105750ec12f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Irtiza. What do you mean by paginate? I'm guessing = you mean doing something like taking the results of a query like

SELECT name, age FROM users SORT BY age DESC

<= div>and displaying the results on some UI 10 at a time, say.

=
If that's the case, the answer is no. It requires additional= application code. In general, Kudu cannot return rows in order. So, if you= want rows 101-110, you must retrieve *all* the rows, select the top 110, a= nd then display only the final 10.=C2=A0

In specia= l cases when the sort is on a prefix of the primary key, scan tokens can be= used to have Kudu return sorted subsets of rows from each tablet, which yo= u can partially merge to get the desired result set.

With a lot of data it's best to retrieve a large amount of sorted re= sults and paginate from the cached results, rather than running a new query= per page.

-Will

On Tue, Sep 4, 2018 at 9:02 AM Irtiza Ali <iali@an10.io> wrote:
Hello everyone,

I= s there a way to paginate kudu's data using its python client?


I
--000000000000855b1105750ec12f--