kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From t...@apache.org
Subject kudu git commit: Fast path scanning blocks of deleted rows
Date Tue, 05 Jun 2018 23:28:56 GMT
Repository: kudu
Updated Branches:
  refs/heads/branch-1.5.x 33334affa -> 81325bf5f

Fast path scanning blocks of deleted rows

This adds some very simple fast-paths in the case that an entire row
block consists of rows that were deleted.

The first is in the block materialization code: if delta application
results in all rows being deleted, we don't need to move forward and
materialize any columns.

The second is before serializing scan responses to the client. In this
case we don't need to loop over each column and read the selection
vector for each column. Instead we can just return immediately.

I tested this on a table where I'd inserted a few billion rows, deleted
them all, and then re-inserted them. Before the patch, a simple 'SELECT
* FROM t LIMIT 10' took 306sec. With the first optimization only, it
took about 10 seconds. With the second one as well, it took about 2.2

There are probably more general optimizations that could be done for
sparsely-populated RowBlocks (eg where just a few rows are selected) but
they are much more complex, and it's relatively common for users to
delete large consecutive runs of rows. For example, users may use
'DELETE' all rows in a table or partition before re-adding them, or they
may delete all data corresponding to some prefix of the PK.

Change-Id: I9fa891c0f4e857ddd1f873ad4855154d078be6b8
Reviewed-on: http://gerrit.cloudera.org:8080/10213
Tested-by: Todd Lipcon <todd@apache.org>
Reviewed-by: Will Berkeley <wdberkeley@gmail.com>
Reviewed-on: http://gerrit.cloudera.org:8080/10328
Tested-by: Will Berkeley <wdberkeley@gmail.com>
Reviewed-by: Andrew Wong <awong@cloudera.com>
Reviewed-on: http://gerrit.cloudera.org:8080/10609
Reviewed-by: Todd Lipcon <todd@apache.org>

Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/81325bf5
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/81325bf5
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/81325bf5

Branch: refs/heads/branch-1.5.x
Commit: 81325bf5f2572e5d3fb7af0e53db7a1b4ac2c500
Parents: 33334af
Author: Todd Lipcon <todd@apache.org>
Authored: Wed Apr 25 15:54:45 2018 -0700
Committer: Will Berkeley <wdberkeley@gmail.com>
Committed: Tue Jun 5 20:17:48 2018 +0000

 src/kudu/common/generic_iterators.cc | 7 +++++++
 src/kudu/tserver/tablet_service.cc   | 8 ++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/src/kudu/common/generic_iterators.cc b/src/kudu/common/generic_iterators.cc
index 76fcb80..40d6151 100644
--- a/src/kudu/common/generic_iterators.cc
+++ b/src/kudu/common/generic_iterators.cc
@@ -524,6 +524,13 @@ Status MaterializingIterator::MaterializeBlock(RowBlock *dst) {
   // been deleted.
+  // It's relatively common to delete large sequential chunks of rows.
+  // We can fast-path that case and avoid reading any column data.
+  if (!dst->selection_vector()->AnySelected()) {
+    DVLOG(1) << "Fast path over " << dst->nrows() << " deleted rows";
+    return Status::OK();
+  }
   for (const auto& col_pred : col_idx_predicates_) {
     // Materialize the column itself into the row block.
     ColumnBlock dst_col(dst->column_block(get<0>(col_pred)));

diff --git a/src/kudu/tserver/tablet_service.cc b/src/kudu/tserver/tablet_service.cc
index 9e1f46a..5ca6cf4 100644
--- a/src/kudu/tserver/tablet_service.cc
+++ b/src/kudu/tserver/tablet_service.cc
@@ -478,8 +478,12 @@ class ScanResultCopier : public ScanResultCollector {
         pad_unixtime_micros_to_16_bytes_(false) {}
   void HandleRowBlock(const Schema* client_projection_schema,
-                              const RowBlock& row_block) override {
-    num_rows_returned_ += row_block.selection_vector()->CountSelected();
+                      const RowBlock& row_block) override {
+    int64_t num_selected = row_block.selection_vector()->CountSelected();
+    // Fast-path empty blocks (eg because the predicate didn't match any rows or
+    // all rows in the block were deleted)
+    if (num_selected == 0) return;
+    num_rows_returned_ += num_selected;
     SerializeRowBlock(row_block, rowblock_pb_, client_projection_schema,
                       rows_data_, indirect_data_, pad_unixtime_micros_to_16_bytes_);
     SetLastRow(row_block, &last_primary_key_);

View raw message