hawq-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From paul...@apache.org
Subject incubator-hawq git commit: HAWQ-1148. Update gtest-parallel to make sure test case can run in both parallel way and serial way
Date Fri, 11 Nov 2016 07:38:04 GMT
Repository: incubator-hawq
Updated Branches:
  refs/heads/master 6f8722a6b -> 6fb913de3


HAWQ-1148. Update gtest-parallel to make sure test case can run in both parallel way and serial
way

Signed-off-by: wengyanqing <wengyanqing@gmail.com>


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/6fb913de
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/6fb913de
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/6fb913de

Branch: refs/heads/master
Commit: 6fb913de3050c09780c111b2bbceb6d93cd99f6a
Parents: 6f8722a
Author: amyrazz44 <abai@pivotal.io>
Authored: Tue Nov 8 16:03:55 2016 +0800
Committer: Paul Guo <paulguo@gmail.com>
Committed: Fri Nov 11 15:37:47 2016 +0800

----------------------------------------------------------------------
 src/test/feature/README.md      |   4 +-
 src/test/feature/gtest-parallel | 154 +++++++++++++++++++++++++++--------
 src/test/feature/schedule.txt   |   6 ++
 src/test/feature/sequence.txt   |   1 -
 4 files changed, 129 insertions(+), 36 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/6fb913de/src/test/feature/README.md
----------------------------------------------------------------------
diff --git a/src/test/feature/README.md b/src/test/feature/README.md
index 85ed5c6..b0bdf13 100644
--- a/src/test/feature/README.md
+++ b/src/test/feature/README.md
@@ -15,8 +15,8 @@ Before building the code of feature tests part, just make sure your compiler
sup
 1. Make sure HAWQ is running correctly. If not, `init` or `start` HAWQ at first. Note please
don't set locale related arguments for hawq init.
 2. Load environment configuration by running `source $INSTALL_PREFIX/greenplum_path.sh`.
 3. Load hdfs configuration. For example, `export HADOOP_HOME=/Users/wuhong/hadoop-2.7.2 &&
export PATH=${PATH}:${HADOOP_HOME}/bin`. Since some test cases need `hdfs` and `hadoop` command,
just ensure these commands work before running. Otherwise you will get failure.
-4. Run the cases with`./parallel-run-feature-test.sh 8 ./feature-test`(in this case 8 threads
in parallel), you could use `--gtest_filter` option to filter test cases(both positive and
negative patterns are supported). Please see more options by running `./feature-test --help`.
-5. As for now, there are several cases which in the sequence.txt file can not parallel run.
Run the cases with `./parallel-run-feature-test.sh 8 ./feature-test --gtest_filter=-`cat ./sequence.txt``.
+4. Run the cases with`./parallel-run-feature-test.sh 8 ./feature-test`(in this case 8 threads
in parallel), you could use `--gtest_filter` option to filter test cases(both positive and
negative patterns are supported). Please see more options by running `./feature-test --help`.

+5.You can also run cases with `./parallel-run-feature-test.sh 8 ./feature-test --gtest_schedule`
(eg. --gtest_schedule=./schedule.txt) if you want to run cases in both parallel way and serial
way.The schedule file sample is schedule.txt which stays in the same directory.
 
 # Development
 In contribution to HAWQ, we suggest developers submitting feature tests related to your feature
development. In writting a featurte test, you need to write a cpp file inside corresponding
folders. There are two recommended way to write this cpp file:

http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/6fb913de/src/test/feature/gtest-parallel
----------------------------------------------------------------------
diff --git a/src/test/feature/gtest-parallel b/src/test/feature/gtest-parallel
index 9b1f9ee..7d8570a 100755
--- a/src/test/feature/gtest-parallel
+++ b/src/test/feature/gtest-parallel
@@ -170,7 +170,7 @@ class FilterFormat:
       self.out.permanent_line("FAILED TESTS (%d/%d):"
                               % (len(self.failures), self.total_tests))
       for (binary, test) in self.failures:
-        self.out.permanent_line(" " + binary + ": " + test)
+        self.out.permanent_line(" " + binary + ": " + test + " [" + test_map[test] + "]")
     self.out.flush_transient_output()
 
 class RawFormat:
@@ -234,6 +234,69 @@ class TestTimes(object):
     except IOError:
       pass  # ignore errors---saving the times isn't that important
 
+
+def get_schedule(schedule_file):
+  "read schedule file from --gtest_schedule option"
+  try:
+    sfile = open(schedule_file, 'r')
+  except (EOFError, IOError):
+    sys.exit("Read file error")
+  
+  filter_test,  ptest, stest = [], [], []
+
+  for line in sfile.readlines():
+    if not line.strip():
+      continue
+    if line[0] == '#':
+      continue
+
+    testline = line.split('=')
+    if len(testline) != 2 or (not testline[1].strip()):
+      sys.exit("format error in schedule file")
+    
+    (key, value) = testline
+    if key == "PARALLEL":
+      ptest.append(value) 
+    elif key == "SERIAL":
+      stest.append(value)
+    else:
+      sys.exit("format error in schedule file")
+
+  filter_test = ptest + stest	
+  sfile.close()
+  return len(ptest), filter_test
+
+
+def do_gtest_filter(list_command, command, op_filter):
+    "get tests by --gtest_filter and --gtest_list_tests"
+    list_command += ['--gtest_filter=' + op_filter]
+    try:
+      test_list = subprocess.Popen(list_command + ['--gtest_list_tests'],
+                                 stdout=subprocess.PIPE).communicate()[0]
+    except OSError as e:
+      sys.exit("%s: %s" % (test_binary, str(e)))
+
+    command += additional_args
+    tests = []
+    test_group = ''
+    for line in test_list.split('\n'):
+      if not line.strip():
+        continue
+      if line[0] != " ":
+        # Remove comments for typed tests and strip whitespace.
+        test_group = line.split('#')[0].strip()
+        continue
+      # Remove comments for parameterized tests and strip whitespace.
+      line = line.split('#')[0].strip()
+      if not line:
+        continue
+      test = test_group + line
+      if not options.gtest_also_run_disabled_tests and 'DISABLED_' in test:
+        continue
+      tests.append((times.get_test_time(test_binary, test),
+                    test_binary, test, command))
+    return tests
+
 # Remove additional arguments (anything after --).
 additional_args = []
 
@@ -260,6 +323,8 @@ parser.add_option('--gtest_color', type='string', default='yes',
                   help='color output')
 parser.add_option('--gtest_filter', type='string', default='',
                   help='test filter')
+parser.add_option('--gtest_schedule', type='string', default='',
+                  help='test schedule which contains parallel and serial tests')
 parser.add_option('--gtest_also_run_disabled_tests', action='store_true',
                   default=False, help='run disabled tests too')
 parser.add_option('--format', type='string', default='filter',
@@ -269,6 +334,8 @@ parser.add_option('--print_test_times', action='store_true', default=False,
 
 (options, binaries) = parser.parse_args()
 
+
+
 if binaries == []:
   parser.print_usage()
   sys.exit(1)
@@ -285,41 +352,42 @@ else:
 save_file = os.path.join(os.path.expanduser("~"), ".gtest-parallel-times")
 times = TestTimes(save_file)
 tests = []
+#pull all the tests into test_map dict, in order to mark the failed test in FAILED log 
+test_map = {} 
+# mark the end of paralell test id by parallel_id
+parallel_id = 0
+
 for test_binary in binaries:
   command = [test_binary]
   if options.gtest_also_run_disabled_tests:
     command += ['--gtest_also_run_disabled_tests']
-
   list_command = list(command)
-  if options.gtest_filter != '':
-    list_command += ['--gtest_filter=' + options.gtest_filter]
-
-  try:
-    test_list = subprocess.Popen(list_command + ['--gtest_list_tests'],
-                                 stdout=subprocess.PIPE).communicate()[0]
-  except OSError as e:
-    sys.exit("%s: %s" % (test_binary, str(e)))
-
-  command += additional_args
-
-  test_group = ''
-  for line in test_list.split('\n'):
-    if not line.strip():
-      continue
-    if line[0] != " ":
-      # Remove comments for typed tests and strip whitespace.
-      test_group = line.split('#')[0].strip()
-      continue
-    # Remove comments for parameterized tests and strip whitespace.
-    line = line.split('#')[0].strip()
-    if not line:
-      continue
-
-    test = test_group + line
-    if not options.gtest_also_run_disabled_tests and 'DISABLED_' in test:
-      continue
-    tests.append((times.get_test_time(test_binary, test),
-                  test_binary, test, command))
+  
+  if options.gtest_schedule and options.gtest_filter :
+    sys.exit("Option input failure : gtest_schedule and gtest_filter can not use in the same
time: \n")
+    
+  if options.gtest_schedule :
+    (pcount, filter_test) = get_schedule(options.gtest_schedule)
+    for i in range(0, len(filter_test)):
+      test = do_gtest_filter(list_command, command, filter_test[i].strip())
+      if i < pcount:
+        parallel_id += len(test)
+      for j in range(0, len(test)):
+        tests.append(test[j])
+        if i < pcount:
+          test_map[test[j][2]] = 'PARALLEL'
+        else:
+          test_map[test[j][2]] = 'SERIAL'
+      
+  if options.gtest_filter :
+    tests = do_gtest_filter(list_command, command, options.gtest_filter)
+    if options.workers > 1:
+      for test in tests:
+        test_map[test[2]] = 'PARALLEL'
+    else:
+      for test in tests:
+        test_map[test[2]] = 'SERIAL'
+    parallel_id = len(tests)
 
 if options.failed:
   # The first element of each entry is the runtime of the most recent
@@ -330,7 +398,8 @@ if options.failed:
 # Sort tests by falling runtime (with None, which is what we get for
 # new and failing tests, being considered larger than any real
 # runtime).
-tests.sort(reverse=True, key=lambda x: ((1 if x[0] is None else 0), x))
+# sort [0..parallel_id]
+tests[:parallel_id].sort(reverse=True, key=lambda x: ((1 if x[0] is None else 0), x))
 
 # Repeat tests (-r flag).
 tests *= options.repeat
@@ -354,6 +423,8 @@ for logfile in os.listdir(options.output_dir):
 # Run the specified job. Return the elapsed time in milliseconds if
 # the job succeeds, or None if the job fails. (This ensures that
 # failing tests will run first the next time.)
+
+
 def run_job((command, job_id, test)):
   begin = time.time()
 
@@ -381,7 +452,7 @@ def worker():
   while True:
     job = None
     test_lock.acquire()
-    if job_id < len(tests):
+    if job_id < parallel_id :
       (_, test_binary, test, command) = tests[job_id]
       logger.log(str(job_id) + ': TEST ' + test_binary + ' ' + test)
       job = (command, job_id, test)
@@ -397,9 +468,26 @@ def start_daemon(func):
   t.start()
   return t
 
+
+#run test in parallel way
 workers = [start_daemon(worker) for i in range(options.workers)]
 
 [t.join() for t in workers]
+
+
+#run test in serail way
+job_id = parallel_id
+while True:
+  job = None
+  if job_id < len(tests):
+    (_, test_binary, test, command) = tests[job_id]
+    logger.log(str(job_id) + ': TEST ' + test_binary + ' ' + test)
+    job = (command, job_id, test)
+  job_id += 1
+  if job is None:
+    break
+  times.record_test_time(test_binary, test, run_job(job))
+
 logger.end()
 times.write_to_file(save_file)
 if options.print_test_times:

http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/6fb913de/src/test/feature/schedule.txt
----------------------------------------------------------------------
diff --git a/src/test/feature/schedule.txt b/src/test/feature/schedule.txt
new file mode 100644
index 0000000..18e7366
--- /dev/null
+++ b/src/test/feature/schedule.txt
@@ -0,0 +1,6 @@
+#PARALLEL=* are the parallel tests to run, optional but should not be empty
+#SERIAL=* are the serial tests to run, optional but should not be empty
+#you can have several PARALLEL or SRRIAL
+
+PARALLEL=TestErrorTable.*:TestExternalTable.*:TestPreparedStatement.*:TestUDF.*:TestAOSnappy.*:TestAlterOwner.*:TestAlterTable.*:TestCreateTable.*:TestGuc.*:TestType.*:TestDatabase.*:TestParquet.*:TestPartition.*:TestSubplan.*:TestAggregate.*:TestCreateTypeComposite.*:TestGpDistRandom.*:TestInformationSchema.*:TestQueryInsert.*:TestQueryNestedCaseNull.*:TestQueryPolymorphism.*:TestQueryPortal.*:TestQueryPrepare.*:TestQuerySequence.*:TestCommonLib.*:TestToast.*:TestTransaction.*:TestCommand.*:TestCopy.*:TestExternalTable.TestExternalTableAll
+SERIAL=TestHawqRegister.*:TestExternalOid.TestExternalOidAll:TestTemp.BasicTest:TestRowTypes.*

http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/6fb913de/src/test/feature/sequence.txt
----------------------------------------------------------------------
diff --git a/src/test/feature/sequence.txt b/src/test/feature/sequence.txt
deleted file mode 100644
index 75cd7b3..0000000
--- a/src/test/feature/sequence.txt
+++ /dev/null
@@ -1 +0,0 @@
-TestHawqRegister.*:TestExternalOid.TestExternalOidAll:TestTemp.BasicTest:TestExternalTable.TestExternalTableAll


Mime
View raw message