hawq-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r...@apache.org
Subject incubator-hawq git commit: HAWQ-984. hawq config is too slow.
Date Mon, 29 Aug 2016 03:03:08 GMT
Repository: incubator-hawq
Updated Branches:
  refs/heads/master f27659303 -> 5e94e3655


HAWQ-984. hawq config is too slow.

Further code change for the issue. Limit the worker thread number
if the queue entry number is less than the default
worker thread number (i.e. 16) since more worker thread number is
totally wasteful.


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/5e94e365
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/5e94e365
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/5e94e365

Branch: refs/heads/master
Commit: 5e94e36559ef88e4e8c035e9c2d1980f09b517d4
Parents: f276593
Author: Paul Guo <paulguo@gmail.com>
Authored: Sat Aug 27 16:22:43 2016 +0800
Committer: rlei <rlei@pivotal.io>
Committed: Mon Aug 29 11:01:41 2016 +0800

----------------------------------------------------------------------
 tools/bin/gppylib/util/ssh_utils.py | 6 +++---
 tools/bin/hawqpylib/hawqlib.py      | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/5e94e365/tools/bin/gppylib/util/ssh_utils.py
----------------------------------------------------------------------
diff --git a/tools/bin/gppylib/util/ssh_utils.py b/tools/bin/gppylib/util/ssh_utils.py
index f42a5d3..6240bed 100644
--- a/tools/bin/gppylib/util/ssh_utils.py
+++ b/tools/bin/gppylib/util/ssh_utils.py
@@ -115,7 +115,7 @@ class HostList():
     def checkSSH(self):
         '''Check that ssh to hostlist is okay.'''
 
-        pool = WorkerPool()
+        pool = WorkerPool(min(len(self.list), 16))
 
         for h in self.list:
             cmd = Echo('ssh test', '', ctxt=REMOTE, remoteHost=h)
@@ -135,7 +135,7 @@ class HostList():
         '''For multiple host that is of the same node, keep only one in the hostlist.'''
         unique = {}
 
-        pool = WorkerPool()
+        pool = WorkerPool(min(len(self.list), 16))
         for h in self.list:
             cmd = Hostname('hostname', ctxt=REMOTE, remoteHost=h)
             pool.addCommand(cmd)
@@ -159,7 +159,7 @@ class HostList():
     def removeBadHosts(self):
         ''' Update list of host to include only the host on which SSH was successful'''
 
-        pool = WorkerPool()
+        pool = WorkerPool(min(len(self.list), 16))
 
         for h in self.list:
             cmd = Echo('ssh test', '', ctxt=REMOTE, remoteHost=h)

http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/5e94e365/tools/bin/hawqpylib/hawqlib.py
----------------------------------------------------------------------
diff --git a/tools/bin/hawqpylib/hawqlib.py b/tools/bin/hawqpylib/hawqlib.py
index c149ffc..d9bd85d 100755
--- a/tools/bin/hawqpylib/hawqlib.py
+++ b/tools/bin/hawqpylib/hawqlib.py
@@ -586,7 +586,7 @@ def get_host_status(hostlist):
     if not isinstance(hostlist, list):
         raise Exception("Input parameter should be of type list")
 
-    pool = WorkerPool()
+    pool = WorkerPool(min(len(hostlist), 16))
 
     for host in hostlist:
         cmd = Echo('ssh test', '', ctxt=REMOTE, remoteHost=host)


Mime
View raw message