Return-Path: Delivered-To: apmail-hadoop-core-commits-archive@www.apache.org Received: (qmail 12644 invoked from network); 30 Jan 2008 23:22:37 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 30 Jan 2008 23:22:37 -0000 Received: (qmail 51110 invoked by uid 500); 30 Jan 2008 23:22:28 -0000 Delivered-To: apmail-hadoop-core-commits-archive@hadoop.apache.org Received: (qmail 51089 invoked by uid 500); 30 Jan 2008 23:22:28 -0000 Mailing-List: contact core-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-commits@hadoop.apache.org Received: (qmail 51080 invoked by uid 99); 30 Jan 2008 23:22:28 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Jan 2008 15:22:28 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO eris.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Jan 2008 23:22:08 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 08F8C1A9832; Wed, 30 Jan 2008 15:22:15 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r616948 - in /hadoop/core/branches/branch-0.16: CHANGES.txt src/contrib/hod/hodlib/GridServices/hdfs.py src/contrib/hod/hodlib/GridServices/mapred.py src/contrib/hod/hodlib/HodRing/hodRing.py Date: Wed, 30 Jan 2008 23:22:14 -0000 To: core-commits@hadoop.apache.org From: nigel@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20080130232215.08F8C1A9832@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: nigel Date: Wed Jan 30 15:22:10 2008 New Revision: 616948 URL: http://svn.apache.org/viewvc?rev=616948&view=rev Log: HADOOP-2740. Modify HOD to work with changes made by in HADOOP-2404. Contributed by Hemanth Yamijala Modified: hadoop/core/branches/branch-0.16/CHANGES.txt hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/GridServices/hdfs.py hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/GridServices/mapred.py hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/HodRing/hodRing.py Modified: hadoop/core/branches/branch-0.16/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.16/CHANGES.txt?rev=616948&r1=616947&r2=616948&view=diff ============================================================================== --- hadoop/core/branches/branch-0.16/CHANGES.txt (original) +++ hadoop/core/branches/branch-0.16/CHANGES.txt Wed Jan 30 15:22:10 2008 @@ -649,6 +649,9 @@ HADOOP-2404. Fix backwards compatability with hadoop-0.15 configuration files that was broken by HADOOP-2185. (omalley) + + HADOOP-2740. Modify HOD to work with changes made by in HADOOP-2404. + (Hemanth Yamijala via nigel) Release 0.15.3 - 2008-01-18 Modified: hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/GridServices/hdfs.py URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/GridServices/hdfs.py?rev=616948&r1=616947&r2=616948&view=diff ============================================================================== --- hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/GridServices/hdfs.py (original) +++ hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/GridServices/hdfs.py Wed Jan 30 15:22:10 2008 @@ -59,7 +59,7 @@ str(self.serviceDesc.dict['info_port']) else: # After Hadoop-2185 - self.serviceDesc.dict['final-attrs']['dfs.http.bindAddress'] = "%s:%s" % \ + self.serviceDesc.dict['final-attrs']['dfs.http.address'] = "%s:%s" % \ (dict['host'], dict['info_port']) def getInfoAddrs(self): @@ -70,7 +70,7 @@ infoaddr = k + ':' + attrs['dfs.info.port'] else: # After Hadoop-2185 - infoaddr = attrs['dfs.http.bindAddress'] + infoaddr = attrs['dfs.http.address'] return [infoaddr] class Hdfs(MasterSlave): @@ -142,7 +142,7 @@ self.infoAddr = self.masterNode + ':' + dict['dfs.info.port'] else: # After Hadoop-2185 - self.infoAddr = dict['dfs.http.bindAddress'] + self.infoAddr = dict['dfs.http.address'] def _parseEquals(self, list): return parseEquals(list) @@ -168,13 +168,13 @@ if 'dfs.info.bindAddress' not in attrs: return ServiceUtil.getUniqPort() else: - if 'dfs.http.bindAddress' not in attrs: + if 'dfs.http.address' not in attrs: return ServiceUtil.getUniqPort() if self.version < 16: p = attrs['dfs.info.port'] else: - p = attrs['dfs.http.bindAddress'].split(':')[1] + p = attrs['dfs.http.address'].split(':')[1] try: return int(p) except: @@ -182,7 +182,7 @@ if self.version < 16: raise ValueError, "Can't find port from attr dfs.info.port: %s" % (p) else: - raise ValueError, "Can't find port from attr dfs.http.bindAddress: %s" % (p) + raise ValueError, "Can't find port from attr dfs.http.address: %s" % (p) def _setWorkDirs(self, workDirs, envs, attrs, parentDirs, subDir): namedir = None @@ -224,10 +224,10 @@ if 'dfs.info.port' not in attrs: attrs['dfs.info.port'] = 'fillinport' else: - # Addressing Hadoop-2815, added the following. Earlier versions don't + # Addressing Hadoop-2185, added the following. Earlier versions don't # care about this - if 'dfs.http.bindAddress' not in attrs: - attrs['dfs.http.bindAddress'] = 'fillinhostport' + if 'dfs.http.address' not in attrs: + attrs['dfs.http.address'] = 'fillinhostport' self._setWorkDirs(workDirs, envs, attrs, parentDirs, 'hdfs-nn') @@ -305,11 +305,11 @@ if 'dfs.datanode.info.port' not in attrs: attrs['dfs.datanode.info.port'] = 'fillinport' else: - # Adding the following. Hadoop-2815 - if 'dfs.datanode.bindAddress' not in attrs: - attrs['dfs.datanode.bindAddress'] = 'fillinhostport' - if 'dfs.datanode.http.bindAddress' not in attrs: - attrs['dfs.datanode.http.bindAddress'] = 'fillinhostport' + # Adding the following. Hadoop-2185 + if 'dfs.datanode.address' not in attrs: + attrs['dfs.datanode.address'] = 'fillinhostport' + if 'dfs.datanode.http.address' not in attrs: + attrs['dfs.datanode.http.address'] = 'fillinhostport' self._setWorkDirs(workDirs, envs, attrs, parentDirs, 'hdfs-dn') Modified: hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/GridServices/mapred.py URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/GridServices/mapred.py?rev=616948&r1=616947&r2=616948&view=diff ============================================================================== --- hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/GridServices/mapred.py (original) +++ hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/GridServices/mapred.py Wed Jan 30 15:22:10 2008 @@ -65,7 +65,7 @@ str(self.serviceDesc.dict['info_port']) else: # After Hadoop-2185 - self.serviceDesc['final-attrs']['mapred.job.tracker.http.bindAddress'] = \ + self.serviceDesc['final-attrs']['mapred.job.tracker.http.address'] = \ "%s:%s" %(dict['host'], dict['info_port']) def getInfoAddrs(self): @@ -76,8 +76,8 @@ infoaddr = k + ':' + attrs['mapred.job.tracker.info.port'] else: # After Hadoop-2185 - # Note: earlier,we never respected mapred.job.tracker.http.bindAddress - infoaddr = attrs['mapred.job.tracker.http.bindAddress'] + # Note: earlier,we never respected mapred.job.tracker.http.address + infoaddr = attrs['mapred.job.tracker.http.address'] return [infoaddr] class MapReduce(MasterSlave): @@ -144,7 +144,7 @@ self.infoAddr = self.masterNode + ':' + dict['mapred.job.tracker.info.port'] else: # After Hadoop-2185 - self.infoAddr = dict['mapred.job.tracker.http.bindAddress'] + self.infoAddr = dict['mapred.job.tracker.http.address'] def _parseEquals(self, list): return parseEquals(list) @@ -171,13 +171,13 @@ if not 'mapred.job.tracker.info.port' in attrs: return ServiceUtil.getUniqPort() else: - if 'mapred.job.tracker.http.bindAddress' not in attrs: + if 'mapred.job.tracker.http.address' not in attrs: return ServiceUtil.getUniqPort() if self.version < 16: p = attrs['mapred.job.tracker.info.port'] else: - p = attrs['mapred.job.tracker.http.bindAddress'].split(':')[1] + p = attrs['mapred.job.tracker.http.address'].split(':')[1] try: return int(p) except: @@ -185,7 +185,7 @@ if self.version < 16: raise ValueError, "Can't find port from attr mapred.job.tracker.info.port: %s" % (p) else: - raise ValueError, "Can't find port from attr mapred.job.tracker.http.bindAddress: %s" % (p) + raise ValueError, "Can't find port from attr mapred.job.tracker.http.address: %s" % (p) def _setWorkDirs(self, workDirs, envs, attrs, parentDirs, subDir): local = [] @@ -232,9 +232,9 @@ if 'mapred.job.tracker.info.port' not in attrs: attrs['mapred.job.tracker.info.port'] = 'fillinport' else: - # Addressing Hadoop-2815, - if 'mapred.job.tracker.http.bindAddress' not in attrs: - attrs['mapred.job.tracker.http.bindAddress'] = 'fillinhostport' + # Addressing Hadoop-2185, + if 'mapred.job.tracker.http.address' not in attrs: + attrs['mapred.job.tracker.http.address'] = 'fillinhostport' attrs['fs.default.name'] = hdfs.getMasterAddrs()[0] @@ -274,11 +274,11 @@ # earlier to 16, tasktrackers always took ephemeral port 0 for # tasktracker.report.bindAddress else: - # Adding the following. Hadoop-2815 - if 'mapred.task.tracker.report.bindAddress' not in attrs: - attrs['mapred.task.tracker.report.bindAddress'] = 'fillinhostport' - if 'mapred.task.tracker.http.bindAddress' not in attrs: - attrs['mapred.task.tracker.http.bindAddress'] = 'fillinhostport' + # Adding the following. Hadoop-2185 + if 'mapred.task.tracker.report.address' not in attrs: + attrs['mapred.task.tracker.report.address'] = 'fillinhostport' + if 'mapred.task.tracker.http.address' not in attrs: + attrs['mapred.task.tracker.http.address'] = 'fillinhostport' self._setWorkDirs(workDirs, envs, attrs, parentDirs, 'mapred-tt') Modified: hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/HodRing/hodRing.py URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/HodRing/hodRing.py?rev=616948&r1=616947&r2=616948&view=diff ============================================================================== --- hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/HodRing/hodRing.py (original) +++ hadoop/core/branches/branch-0.16/src/contrib/hod/hodlib/HodRing/hodRing.py Wed Jan 30 15:22:10 2008 @@ -564,7 +564,7 @@ jettyAddr = jobTrackerAddr.split(':')[0] + ':' + \ attrs['mapred.job.tracker.info.port'] else: - jettyAddr = attrs['mapred.job.tracker.http.bindAddress'] + jettyAddr = attrs['mapred.job.tracker.http.address'] self.log.debug("Jobtracker jetty : %s" % jettyAddr) # Check for Jetty to come up