Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of driddle@gmail.com designates
 209.85.146.178 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition;
        b=flHsJVc4IVCzdz3Olz85dsLhvN3DnrVaX3kIsUqhhQDcfGJmQ7NNgTYzQz1AjhGqyeLRkDfUUB8ujbYDlMlqLyXJyilNG0uft5arWcFACucUMZ6Bu0HJV4p7v/gbA8VrGXalQYJLzuqrEkieEVy00YGb44C+JCplrVjQqgj8AcA=
Message-ID: <ec825f050804291502j126b5db1rbb4c5ead4e35aefa@mail.gmail.com>
Date: Tue, 29 Apr 2008 17:02:36 -0500
From: "David Riddle" <driddle@gmail.com>
To: core-user@hadoop.apache.org
Subject: HOD and Condor
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi,

I am trying to setup Hadoop on Demand on a cluster running Condor. I
downloaded  hod-open-4.tar.gz from here:

http://issues.apache.org/jira/browse/HADOOP-1301

First of all I am not sure if this is the most recent version to use
or if this the appropriate place to retrieve it. The website for
Hadoop does not seem to have a download link for hod even though it
contains documentation for hod.

The version I download seems to have some support for condor but it is
not documented. I tryed to setup the hodrc as best I could but with
not much sucesses. I have included a copy of my config file below if
someone could help me to get hod working I would be grateful. When I
run the following command:

./hod -c ../conf/hodrc -m 3 -a "run dfs -ls /"

I recieve the following errors:

Failed to start hod shell: <type 'exceptions.AttributeError'>
NodePoolDesc instance has no attribute 'getPkgDirs'


Traceback (most recent call last):
  File "hod", line 463, in <module>
    exitStatus = main(hodConfig)
  File "/home/ugrad/ddriddle/src/hod/hodlib/Hod/hod.py", line 120, in main
    if np: np.finalize()
  File "/home/ugrad/ddriddle/src/hod/hodlib/NodePools/condor.py", line
73, in finalize
    self.freeNodeSet(ns)
  File "/home/ugrad/ddriddle/src/hod/hodlib/NodePools/condor.py", line
64, in freeNodeSet
    self._delNodeSet(nodeSet)
  File "/home/ugrad/ddriddle/src/hod/hodlib/NodePools/condor.py", line
219, in _delNodeSet
    pkgdirs = npd.getPkgDirs()
AttributeError: NodePoolDesc instance has no attribute 'getPkgDirs'


Thanks,

David Riddle

# Template hodrc file
#

# -- hod shell specific configuration
[hod]
temp-dir             = /tmp/hod-ddriddle
#java-home            = ${JAVA_HOME}
java-home            = /home/ugrad/ddriddle/lib/java
debug                = 3
stream               = True
non-interactive      = false
allocate-timeout     = 3600
#cluster              = ${CLUSTER_NAME}
cluster              = hal9000
#hadoop-install       = ${HADOOP_HOME}
hadoop-install       = /home/ugrad/ddriddle/src/hadoop-0.15.3
#log-dir              = /var/hod/logs-${RM_USER}
#log-collection-dir   = /var/hod/logs-${RM_USER}
#syslog-address       = localhost:8814
#cluster-factor       = 1.8

# -- resource manager configuration
[resource_manager]
#id                  = ${RM_ID}
id                  = condor
#pbs-user            = ${RM_USER}
pbs-user            = ddriddle
pbs-server          = hal9000-server.cs.uiuc.edu
queue               = parallel
batch-home          = /home/hal/condor/condor
env-vars            = HOD_PYTHON_HOME=/home/ugrad/ddriddle/bin

# -- serive_registry configuration
[service_registry]
debug                = 2
http-address         = :8002
xrs-address          = :8879
#stream               = False
#log-dir              = /var/hod/logs-${RM_USER}
#xrs-port-range     = 32768-65536

# -- ringmaster configuration
[ringmaster]
temp-dir             = /tmp/hod-ddriddle
work-dirs            =
/tmp/hod-ddriddle/1,/tmp/hod-ddriddle/2,/tmp/hod-ddriddle/3,/tmp/hod-ddriddle/4
register             = true
xrs-port-range       = 32768-65536
http-port-range      = 8000-9000
debug                = 2
#stream               = False
#syslog-address       = localhost:8814
#log-dir              = /var/hod/logs-${RM_USER}
#svcrgy-addr          = :8879

# -- hod ring configuration
[hodring]
temp-dir             = /tmp/hod-ddriddle
register             = true
#java-home            = ${JAVA_HOME}
java-home            = /home/ugrad/ddriddle/lib/java
debug                = 2
xrs-port-range       = 32768-65536
http-port-range      = 8000-9000
#log-dir              = /var/hod/logs-${RM_USER}
#log-destination-uri  = local://var/log-${RM_USER}
#stream               = false
#syslog-address       = localhost:8814
#svcrgy-addr          = :8879

[gridservice-mapred]
external             = false
tracker_port         = 8030
info_port            = 50080
#pkgs                 = ${HADOOP_HOME}
pkgs                 = /home/ugrad/ddriddle/src/hadoop-0.15.3/
#host                 = localhost

[gridservice-hdfs]
external            = false
fs_port             = 8020
info_port           = 50070
pkgs                 = /home/ugrad/ddriddle/src/hadoop-0.15.3/
#pkgs                = ${HADOOP_HOME}
#host                 = localhost

-- 
Always code as if the (person) who ends up maintaining your code will
be a violent psychopath who knows where you live.
 - M. Golding