kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From granthe...@apache.org
Subject [2/4] kudu git commit: [tools] Add table name filter to kudu fs list
Date Fri, 21 Sep 2018 13:45:34 GMT
[tools] Add table name filter to kudu fs list

Due to the use of '--table_name' in 'kudu perf loadgen', I had to move
the definition of the flag to tool_action_common.cc and override the
description for the loadgen flag.

There's no unit tests for the filtering functionality, and I didn't add
any. I did test manually on a cluster with two tables that
'kudu fs list' works as expected with no table name filter, a table name
filter that matches no tables, and a table name filter that matches one
table. I also checked that 'kudu perf loadgen' still works as expected
when `--table_name' is supplied and when it is not supplied.

Change-Id: Iaed1aa0a566a7bfed3fac2574c0da1651059e9d0
Reviewed-on: http://gerrit.cloudera.org:8080/11487
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin <aserbin@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/e48157a0
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/e48157a0
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/e48157a0

Branch: refs/heads/master
Commit: e48157a0e6d1d4d3ee5fb700e020f0c648ff0560
Parents: 3a033d8
Author: Will Berkeley <wdberkeley@gmail.org>
Authored: Thu Sep 20 11:29:41 2018 -0700
Committer: Will Berkeley <wdberkeley@gmail.com>
Committed: Thu Sep 20 22:29:08 2018 +0000

----------------------------------------------------------------------
 src/kudu/tools/tool_action_common.cc |  2 ++
 src/kudu/tools/tool_action_fs.cc     |  9 ++++++++-
 src/kudu/tools/tool_action_perf.cc   | 28 +++++++++++++++-------------
 3 files changed, 25 insertions(+), 14 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/e48157a0/src/kudu/tools/tool_action_common.cc
----------------------------------------------------------------------
diff --git a/src/kudu/tools/tool_action_common.cc b/src/kudu/tools/tool_action_common.cc
index ecc06a5..2005059 100644
--- a/src/kudu/tools/tool_action_common.cc
+++ b/src/kudu/tools/tool_action_common.cc
@@ -88,6 +88,8 @@ DEFINE_string(print_entries, "decoded",
               "  true|1|yes|decoded = print them decoded\n"
               "  pb = print the raw protobuf\n"
               "  id = print only their ids");
+DEFINE_string(table_name, "",
+              "Restrict output to a specific table by name");
 DEFINE_int64(timeout_ms, 1000 * 60, "RPC timeout in milliseconds");
 DEFINE_int32(truncate_data, 100,
              "Truncate the data fields to the given number of bytes "

http://git-wip-us.apache.org/repos/asf/kudu/blob/e48157a0/src/kudu/tools/tool_action_fs.cc
----------------------------------------------------------------------
diff --git a/src/kudu/tools/tool_action_fs.cc b/src/kudu/tools/tool_action_fs.cc
index f23ef81..5da0319 100644
--- a/src/kudu/tools/tool_action_fs.cc
+++ b/src/kudu/tools/tool_action_fs.cc
@@ -88,7 +88,8 @@ DEFINE_bool(repair, false,
             "Repair any inconsistencies in the filesystem.");
 
 DEFINE_string(table_id, "",
-              "Restrict output to a specific table");
+              "Restrict output to a specific table by id");
+DECLARE_string(table_name);
 DEFINE_string(tablet_id, "",
               "Restrict output to a specific tablet");
 DEFINE_int64(rowset_id, -1,
@@ -728,6 +729,7 @@ Status List(const RunnerContext& /*context*/) {
     RETURN_NOT_OK(fs_manager.ListTabletIds(&tablet_ids));
   }
 
+  string table_name = FLAGS_table_name;
   string table_id = FLAGS_table_id;
   ToLowerCase(&table_id);
 
@@ -739,6 +741,10 @@ Status List(const RunnerContext& /*context*/) {
     RETURN_NOT_OK(TabletMetadata::Load(&fs_manager, tablet_id, &tablet_metadata));
     const TabletMetadata& tablet = *tablet_metadata.get();
 
+    if (!table_name.empty() && table_name != tablet.table_name()) {
+      continue;
+    }
+
     if (!table_id.empty() && table_id != tablet.table_id()) {
       continue;
     }
@@ -892,6 +898,7 @@ unique_ptr<Mode> BuildFsMode() {
       .AddOptionalParameter("fs_metadata_dir")
       .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("table_id")
+      .AddOptionalParameter("table_name")
       .AddOptionalParameter("tablet_id")
       .AddOptionalParameter("rowset_id")
       .AddOptionalParameter("column_id")

http://git-wip-us.apache.org/repos/asf/kudu/blob/e48157a0/src/kudu/tools/tool_action_perf.cc
----------------------------------------------------------------------
diff --git a/src/kudu/tools/tool_action_perf.cc b/src/kudu/tools/tool_action_perf.cc
index 63de462..d162bbf 100644
--- a/src/kudu/tools/tool_action_perf.cc
+++ b/src/kudu/tools/tool_action_perf.cc
@@ -174,7 +174,9 @@
 #include <unordered_map>
 #include <vector>
 
+#include <boost/optional/optional.hpp>
 #include <gflags/gflags.h>
+#include <gflags/gflags_declare.h>
 #include <glog/logging.h>
 
 #include "kudu/client/client.h"
@@ -295,18 +297,7 @@ DEFINE_string(auto_database, "default",
               "not created. This flag is useful primarily when the Hive Metastore "
               "integration is enabled in the cluster. If empty, no database is "
               "used.");
-DEFINE_string(table_name, "",
-              "Name of an existing table to use for the test. The test will "
-              "determine the structure of the table schema and "
-              "populate it with data accordingly. If left empty, "
-              "the test automatically creates a table of pre-defined columnar "
-              "structure with unique name and uses it to insert "
-              "auto-generated data. The auto-created table is dropped "
-              "upon successful completion of the test if not overridden "
-              "by the '--keep_auto_table' flag. If running the test against "
-              "an already existing table, it's highly recommended to use a "
-              "dedicated table created just for testing purposes: "
-              "the existing table nor its data is never dropped/deleted.");
+DECLARE_string(table_name);
 DEFINE_int32(table_num_hash_partitions, 8,
              "The number of hash partitions to create when this tool creates "
              "a new table. Note: The total number of partitions must be "
@@ -779,7 +770,18 @@ unique_ptr<Mode> BuildPerfMode() {
       .AddOptionalParameter("show_first_n_errors")
       .AddOptionalParameter("string_fixed")
       .AddOptionalParameter("string_len")
-      .AddOptionalParameter("table_name")
+      .AddOptionalParameter("table_name", boost::none, string(
+            "Name of an existing table to use for the test. The test will "
+            "determine the structure of the table schema and "
+            "populate it with data accordingly. If left empty, "
+            "the test automatically creates a table of pre-defined columnar "
+            "structure with unique name and uses it to insert "
+            "auto-generated data. The auto-created table is dropped "
+            "upon successful completion of the test if not overridden "
+            "by the '--keep_auto_table' flag. If running the test against "
+            "an already existing table, it's highly recommended to use a "
+            "dedicated table created just for testing purposes: "
+            "the existing table nor its data is never dropped/deleted."))
       .AddOptionalParameter("table_num_hash_partitions")
       .AddOptionalParameter("table_num_range_partitions")
       .AddOptionalParameter("table_num_replicas")


Mime
View raw message