Return-Path: Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: (qmail 30829 invoked from network); 4 Mar 2011 01:03:56 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 4 Mar 2011 01:03:56 -0000 Received: (qmail 1897 invoked by uid 500); 4 Mar 2011 01:03:56 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 1869 invoked by uid 500); 4 Mar 2011 01:03:56 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 1861 invoked by uid 99); 4 Mar 2011 01:03:56 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Mar 2011 01:03:56 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Mar 2011 01:03:52 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 59DC523889B2; Fri, 4 Mar 2011 01:03:31 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1076910 - in /hadoop/common/branches/branch-0.20-security-patches/src: contrib/hod/bin/ contrib/hod/conf/ contrib/hod/hodlib/HodRing/ contrib/hod/testing/ docs/src/documentation/content/xdocs/ Date: Fri, 04 Mar 2011 01:03:31 -0000 To: common-commits@hadoop.apache.org From: omalley@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20110304010331.59DC523889B2@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: omalley Date: Fri Mar 4 01:03:30 2011 New Revision: 1076910 URL: http://svn.apache.org/viewvc?rev=1076910&view=rev Log: commit 2dca6e4024b79da6b8503d634c1215eeafd7bc1a Author: Lee Tucker Date: Thu Jul 30 17:40:12 2009 -0700 Applying patch 2532950.2898.patch Modified: hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/hod hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/hodring hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/ringmaster hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/conf/hodrc hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/hodlib/HodRing/hodRing.py hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/testing/testHodRing.py hadoop/common/branches/branch-0.20-security-patches/src/docs/src/documentation/content/xdocs/hod_config_guide.xml Modified: hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/hod URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/hod?rev=1076910&r1=1076909&r2=1076910&view=diff ============================================================================== --- hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/hod (original) +++ hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/hod Fri Mar 4 01:03:30 2011 @@ -350,6 +350,9 @@ defList = { 'hod' : ( ('http-port-range', 'range', 'HTTP port range n-m.', False, None, True, True), + ('hadoop-port-range', 'range', 'Hadoop port range n-m.', + False, None, True, True), + ('service-id', 'string', 'Service ID.', False, None, False, True), Modified: hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/hodring URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/hodring?rev=1076910&r1=1076909&r2=1076910&view=diff ============================================================================== --- hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/hodring (original) +++ hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/hodring Fri Mar 4 01:03:30 2011 @@ -107,6 +107,9 @@ defList = { 'hodring' : ( ('http-port-range', 'range', 'HTTP port range n-m.', False, None, True, True), + + ('hadoop-port-range', 'range', 'Hadoop port range n-m.', + False, None, True, True), ('command', 'string', 'Command for hodring to run.', False, None, False, True), Modified: hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/ringmaster URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/ringmaster?rev=1076910&r1=1076909&r2=1076910&view=diff ============================================================================== --- hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/ringmaster (original) +++ hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/bin/ringmaster Fri Mar 4 01:03:30 2011 @@ -244,6 +244,9 @@ defList = { 'ringmaster' : ( ('http-port-range', 'range', 'HTTP port range n-m.', False, None, True, True), + + ('hadoop-port-range', 'range', 'Hadoop port range n-m.', + False, None, True, True), ('command', 'string', 'Command for hodring to run.', False, None, False, True), Modified: hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/conf/hodrc URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/conf/hodrc?rev=1076910&r1=1076909&r2=1076910&view=diff ============================================================================== --- hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/conf/hodrc (original) +++ hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/conf/hodrc Fri Mar 4 01:03:30 2011 @@ -24,6 +24,7 @@ register = True java-home = ${JAVA_HOME} http-port-range = 8000-9000 xrs-port-range = 32768-65536 +hadoop-port-range = 50000-60000 debug = 3 [resource_manager] Modified: hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/hodlib/HodRing/hodRing.py URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/hodlib/HodRing/hodRing.py?rev=1076910&r1=1076909&r2=1076910&view=diff ============================================================================== --- hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/hodlib/HodRing/hodRing.py (original) +++ hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/hodlib/HodRing/hodRing.py Fri Mar 4 01:03:30 2011 @@ -210,7 +210,7 @@ def createMRSystemDirectoryManager(dict, class HadoopCommand: """Runs a single hadoop command""" - def __init__(self, id, desc, tempdir, tardir, log, javahome, + def __init__(self, id, desc, tempdir, tardir, hadoopportrange, log, javahome, mrSysDir, restart=False): self.desc = desc self.log = log @@ -230,6 +230,7 @@ class HadoopCommand: self.child = None self.restart = restart self.filledInKeyVals = [] + self.__hadoopPortRange = hadoopportrange self._createWorkDirs() self._createHadoopSiteXml() self._createHadoopLogDir() @@ -253,8 +254,9 @@ class HadoopCommand: def createXML(self, doc, attr, topElement, final): for k,v in attr.iteritems(): self.log.debug('_createHadoopSiteXml: ' + str(k) + " " + str(v)) + lowport, highport = self.__hadoopPortRange if ( v == "fillinport" ): - v = "%d" % (ServiceUtil.getUniqRandomPort(low=50000, log=self.log)) + v = "%d" % (ServiceUtil.getUniqRandomPort(low=lowport, high=highport, log=self.log)) keyvalpair = '' if isinstance(v, (tuple, list)): @@ -270,7 +272,7 @@ class HadoopCommand: self.filledInKeyVals.append(keyvalpair) if ( v == "fillinhostport"): - port = "%d" % (ServiceUtil.getUniqRandomPort(low=50000, log=self.log)) + port = "%d" % (ServiceUtil.getUniqRandomPort(low=lowport, high=highport, log=self.log)) self.log.debug('Setting hostname to: %s' % local_fqdn()) v = local_fqdn() + ':' + port @@ -613,7 +615,7 @@ class HodRing(hodBaseService): mrSysDir = getMapredSystemDirectory(self._cfg['mapred-system-dir-root'], self._cfg['userid'], self._cfg['service-id']) self.log.debug('mrsysdir is %s' % mrSysDir) - cmd = HadoopCommand(id, desc, self.__tempDir, self.__pkgDir, self.log, + cmd = HadoopCommand(id, desc, self.__tempDir, self.__pkgDir, self._cfg['hadoop-port-range'], self.log, self._cfg['java-home'], mrSysDir, restart) self.__hadoopLogDirs.append(cmd.logdir) Modified: hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/testing/testHodRing.py URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/testing/testHodRing.py?rev=1076910&r1=1076909&r2=1076910&view=diff ============================================================================== --- hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/testing/testHodRing.py (original) +++ hadoop/common/branches/branch-0.20-security-patches/src/contrib/hod/testing/testHodRing.py Fri Mar 4 01:03:30 2011 @@ -68,7 +68,7 @@ class test_HadoopCommand(unittest.TestCa # that pkgdir is specified at the time of object creation. # END OF TODO self.hadoopCommand = HadoopCommand(self.id, self.desc, self.tempDir, - self.pkgDir, self.log, self.javaHome, + self.pkgDir, (50000, 60000), self.log, self.javaHome, self.mrSysDir, restart=True) self.hadoopSite = os.path.join( self.hadoopCommand.confdir, 'hadoop-site.xml') Modified: hadoop/common/branches/branch-0.20-security-patches/src/docs/src/documentation/content/xdocs/hod_config_guide.xml URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-patches/src/docs/src/documentation/content/xdocs/hod_config_guide.xml?rev=1076910&r1=1076909&r2=1076910&view=diff ============================================================================== --- hadoop/common/branches/branch-0.20-security-patches/src/docs/src/documentation/content/xdocs/hod_config_guide.xml (original) +++ hadoop/common/branches/branch-0.20-security-patches/src/docs/src/documentation/content/xdocs/hod_config_guide.xml Fri Mar 4 01:03:30 2011 @@ -315,6 +315,10 @@ be used by HOD to upload logs if a HDFS URL is specified in log-destination-uri option. Note that this is useful if the users are using a tarball whose version may differ from the external, static HDFS version. + +
  • hadoop-port-range: Range of ports, among which an available port shall + be picked for use to run a Hadoop Service, like JobTracker or TaskTracker.
    • +