incubator-tashi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Stroucken <mxs+apa...@cmu.edu>
Subject Re: Making a release
Date Thu, 26 Jan 2012 00:04:11 GMT
Hi again,

I did another install run, using the findings I posted yesterday. The 
ones I quote are the one's I used:-

> 1)
> A successful install will have prerequisites. These run from low to 
> medium complexity, probably have no generally accepted defaults, and 
> as such I propose we declare them as out of scope for the installation 
> of Tashi itself. Where appropriate, we should give helpful hints 
> though. My list:-
> a) RPyC-3.1.0 is a prerequisite. The software is easily obtained and 
> installed.
> b) A hypervisor is a prerequisite. Qemu is more tested, Xen less so. 
> Tutorials I write will use Qemu for now.
> c) OS images are prerequisites. They need to be prepared before 
> deployment.
> d) A host's hostname should be set (but need not match DNS).
> e) Networking must be engineered beforehand. Tashi will call a script 
> based on network ID; the user should provide that script to connect 
> the hosts virtual interface to the appropriate host network. The host 
> OS must be set up to route if appropriate.
prereqs were applied before the install
>
> 2)
> The default qemuBin configuration points to what appears to be a 
> locally compiled version. This should be set to /usr/bin/kvm. Can 
> someone confirm that /usr/bin/kvm is also the proper name on Ubuntu?
qemuBin was changed to /usr/bin/kvm
>
> 3)
> The NM no longer has use for the infoFile parameter, so it should be 
> removed.
removed
>
> 6)
> Users must change Vfs/prefix to point to somewhere (large) where Tashi 
> can read and write stuff. OS images must be located under ./images. 
> Suspend and resume images must be located under ./suspend.
>
changed to be /tmp
> 8)
> When installing in a place like /usr/local/tashi, running the Tashi 
> programs requires being in that directory and calling the programs 
> like bin/tashi-client.py. This permits the default config to be read, 
> since ./etc is a search path. I would argue that no relative paths 
> should be read by default and ./etc should be changed to 
> /usr/local/tashi/etc. A better thing would be to have the install 
> location configurable.
>
changed to be /usr/local/tashi/etc
> 9)
> The programs all end with ".py". Yes, they are python scripts, but 
> users probably don't care and it may trigger negative associations of 
> scripts. I propose that executables in the bin/ directory be stripped 
> of their suffix.
>
removed .py suffix

Install was done using a grml live-cd Linux install running under a VM 
on a Macbook Pro.
Special notes:-
(XXXstroucki add prereqs here)
(XXXstroucki: the output of createVm is verbose and not generally useful 
to the end user)
(XXXstroucki: the error output here is harmless, but should not happen 
(and does not happen in my cluster). I will address next.)
(XXXstroucki: the leaked stuff comes from lvm2 and can be suppressed by 
setting the environment value LVM_SUPPRESS_FD_WARNINGS. Current attitude 
is for lvm2 to no longer display the messages by default.)
(XXXstroucki: The lvremove output should be quieted.)

Here's the script:-
Prerequisites from the previous message to tashi-dev@a.i.o must be met.
(XXXstroucki add prereqs here)

Script started on Wed 25 Jan 2012 07:47:56 AM CET

root@grml:~# mkdir /tmp/tashi
root@grml:~# cd /tmp/tashi

root@grml:/tmp/tashi# svn co 
http://svn.apache.org/repos/asf/incubator/tashi/branches/stroucki-accounting 
./tashi

A    tashi/NOTICE
A    tashi/LICENSE
A    tashi/doc
A    tashi/doc/DEVELOPMENT
A    tashi/src
...
A    tashi/DISCLAIMER
A    tashi/Makefile
A    tashi/README
 U   tashi
Checked out revision 1235865.

Move the distribution files to their destination (here: /usr/local/tashi)


root@grml:/tmp/tashi# mv tashi /usr/local/tashi
root@grml:/tmp/tashi# cd /usr/local/tashi
root@grml:/usr/local/tashi# ls
DISCLAIMER  doc  etc  LICENSE  Makefile  NOTICE  README  src

Create daemons and clients

root@grml:/usr/local/tashi# make
Symlinking in clustermanager...
Symlinking in nodemanager...
Symlinking in tashi-client...
Symlinking in primitive...
Symlinking in zoni-cli...
Symlinking in Accounting server...
Done
root@grml:/usr/local/tashi# ls bin
accounting  clustermanager  nmd  nodemanager  primitive  tashi-client  
zoni-cli

If the Tashi source directory is not part of the system python search 
path, you must add it.

root@grml:/usr/local/tashi# export PYTHONPATH=$PYTHONPATH:`pwd`/src

Start the cluster manager in debug mode to add a host (in this case, itself)

root@grml:/usr/local/tashi# cd bin
root@grml:/usr/local/tashi/bin# DEBUG=1 ./clustermanager
2012-01-25 07:50:50,285 [./clustermanager:INFO] Using configuration 
file(s) ['/usr/local/tashi/etc/TashiDefaults.cfg']
2012-01-25 07:50:50,286 [./clustermanager:INFO] Starting cluster manager
**********************************************************************
Welcome to IPython. I will try to create a personal configuration directory
where you can customize many aspects of IPython's functionality in:

/root/.ipython
Initializing from configuration: 
/usr/lib/python2.6/dist-packages/IPython/UserConfig

Successful installation!

Please read the sections 'Initial Configuration' and 'Quick Tips' in the
IPython manual (there are both HTML and PDF versions supplied with the
distribution) to make sure that your system environment is properly 
configured
to take advantage of IPython's features.

Important note: the configuration system has changed! The old system is
still in place, but its setting may be partly overridden by the settings in
"~/.ipython/ipy_user_conf.py" config file. Please take a look at the file
if some of the new settings bother you.


Please press <RETURN> to start IPython.
**********************************************************************

from tashi.rpycservices.rpyctypes import Host, HostState, Network
In [1]: from tashi.rpycservices.rpyctypes import Host, HostState, Network

In [2]: data.baseDataObject.hosts[1] = 
Host(d={'id':1,'name':'grml','state': HostState.Normal,'up':False})

In [3]: 
data.baseDataObject.networks[1]=Network(d={'id':272,'name':'default'})

In [4]: data.baseDataObject.save()

In [4]: %Exit

Run the cluster manager in the background:

root@grml:/usr/local/tashi/bin# ./clustermanager &
[1] 4289
root@grml:/usr/local/tashi/bin# 2012-01-25 07:53:43,177 
[./clustermanager:INFO] Using configuration file(s) 
['/usr/local/tashi/etc/TashiDefaults.cfg']
2012-01-25 07:53:43,177 [./clustermanager:INFO] Starting cluster manager

Run the node manager in the background. Note that the hostname must be 
registered with the cluster manager, as shown above.

root@grml:/usr/local/tashi/bin# ./nodemanager &
[2] 4293
root@grml:/usr/local/tashi/bin# 2012-01-25 07:53:59,348 [__main__:INFO] 
Using configuration file(s) ['/usr/local/tashi/etc/TashiDefaults.cfg', 
'/usr/local/tashi/etc/NodeManager.cfg']
2012-01-25 07:53:59,392 
[/usr/local/tashi/src/tashi/nodemanager/vmcontrol/qemu.py:INFO] No VM 
information found in /var/tmp/VmControlQemu/
2012-01-25 07:53:59,404 
[/usr/local/tashi/src/tashi/nodemanager/vmcontrol/qemu.py:INFO] Waiting 
for NM initialization

Verify that the node is shown as being "Up".

root@grml:/usr/local/tashi/bin# ./tashi-client gethosts
 id reserved name decayed up   state  version memory cores notes
----------------------------------------------------------------
 1  []       grml True    True Normal HEAD    233    1     None


Start the primitive scheduling agent:
root@grml:/usr/local/tashi/bin# ./primitive &
[3] 4312

Verify that the cluster manager has full communication with the host. 
When this has happened, decayed is False.

root@grml:/usr/local/tashi/bin# tashi-client gethosts
 id reserved name decayed up   state  version memory cores notes
----------------------------------------------------------------
 1  []       grml False   True Normal HEAD    233    1     None

Check the presence of a disk image:

root@grml:/usr/local/tashi/bin# ls /tmp/images/
debian-wheezy-amd64.qcow2
root@grml:/usr/local/tashi/bin# ./tashi-client getimages
 id imageName                 imageSize
---------------------------------------
 0  debian-wheezy-amd64.qcow2 1.74G   

Create a VM with 1 core and 128 MB of memory using our disk image in 
non-persistent mode:
(XXXstroucki: the output of createVm is verbose and not generally useful 
to the end user)
(XXXstroucki: the error output here is harmless, but should not happen 
(and does not happen in my cluster). I will address next.)

root@grml:/usr/local/tashi/bin# ./tashi-client createVm --cores 1 
--memory 128 --name wheezy --disks debian-wheezy-amd64.qcow2
{
    hostId: None
    name: wheezy
    vmId: None
    decayed: False
    disks: [
        {'uri': 'debian-wheezy-amd64.qcow2', 'persistent': False}
    ]
    userId: 0
    groupName: None
    state: Pending
    nics: [
        {'ip': None, 'mac': '52:54:00:8d:f0:36', 'network': 0}
    ]
    memory: 128
    cores: 1
    id: 1
    hints: {}
}
root@grml:/usr/local/tashi/bin# 2012-01-25 07:56:58,670 
[./primitive:INFO] Scheduling instance wheezy (128 mem, 1 cores, 0 uid) 
on host grml
2012-01-25 07:56:58,679 
[/usr/local/tashi/src/tashi/nodemanager/vmcontrol/qemu.py:INFO] 
Executing command: /usr/bin/kvm   -clock dynticks -drive 
file=/tmp/images/debian-wheezy-amd64.qcow2,if=ide,index=0,cache=off,snapshot=on  
-net nic,macaddr=52:54:00:8d:f0:36,model=virtio,vlan=0 -net 
tap,ifname=tashi1.0,vlan=0,script=/etc/qemu-ifup.0  -m 128 -smp 1 
-serial null -vnc none -monitor pty
2012-01-25 07:56:58,688 
[/usr/local/tashi/src/tashi/nodemanager/vmcontrol/qemu.py:INFO] Adding 
vmId 4370
2012-01-25 07:56:59,104 
[/usr/local/tashi/src/tashi/nodemanager/vmcontrol/qemu.py:ERROR] 
vmStateChange failed for VM wheezy
Traceback (most recent call last):
  File "/usr/local/tashi/src/tashi/nodemanager/vmcontrol/qemu.py", line 
227, in __matchHostPids
    self.nm.vmStateChange(vmId, None, InstanceState.Running)
  File "/usr/local/tashi/src/tashi/nodemanager/nodemanagerservice.py", 
line 237, in vmStateChange
    instance = self.__getInstance(vmId)
  File "/usr/local/tashi/src/tashi/nodemanager/nodemanagerservice.py", 
line 228, in __getInstance
    raise TashiException(d={'errno':Errors.NoSuchVmId,'msg':"There is no 
vmId %d on this host" % (vmId)})
TashiException: {'msg': 'There is no vmId 4370 on this host', 'errno': 3}
2012-01-25 07:57:00,110 
[/usr/local/tashi/src/tashi/nodemanager/vmcontrol/qemu.py:ERROR] 
vmStateChange failed for VM wheezy
Traceback (most recent call last):
  File "/usr/local/tashi/src/tashi/nodemanager/vmcontrol/qemu.py", line 
227, in __matchHostPids
    self.nm.vmStateChange(vmId, None, InstanceState.Running)
  File "/usr/local/tashi/src/tashi/nodemanager/nodemanagerservice.py", 
line 237, in vmStateChange
    instance = self.__getInstance(vmId)
  File "/usr/local/tashi/src/tashi/nodemanager/nodemanagerservice.py", 
line 228, in __getInstance
    raise TashiException(d={'errno':Errors.NoSuchVmId,'msg':"There is no 
vmId %d on this host" % (vmId)})
TashiException: {'msg': 'There is no vmId 4370 on this host', 'errno': 3}

Verify the machine is running:

root@grml:/usr/local/tashi/bin# ./tashi-client getinstances
 id hostId name   user state   disk                      memory cores
---------------------------------------------------------------------
 1  1      wheezy root Running debian-wheezy-amd64.qcow2 128    1   

After the machine had a chance to boot, find out what address it got. If 
you have a DHCP server on your network, search the pool of addresses:

root@grml:/usr/local/tashi/bin# ifconfig br0
br0       Link encap:Ethernet  HWaddr 00:0c:29:62:b3:76 
          inet addr:192.168.244.131  Bcast:192.168.244.255  
Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe62:b376/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2622 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1598 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:730925 (713.7 KiB)  TX bytes:226530 (221.2 KiB)

root@grml:/usr/local/tashi/bin# arp-scan -I br0 192.168.244.0/24
Interface: br0, datalink type: EN10MB (Ethernet)
Starting arp-scan 1.6 with 256 hosts 
(http://www.nta-monitor.com/tools/arp-scan/)
192.168.244.1   00:50:56:c0:00:08       VMWare, Inc.
192.168.244.2   00:50:56:e6:2e:0e       VMWare, Inc.
192.168.244.136 52:54:00:8d:f0:36       QEMU
192.168.244.254 00:50:56:fc:50:42       VMWare, Inc.

4 packets received by filter, 0 packets dropped by kernel
Ending arp-scan 1.6: 256 hosts scanned in 1.493 seconds (171.47 
hosts/sec).  4 responded

Log into the VM:

root@grml:/usr/local/tashi/bin# ssh root@192.168.244.136
The authenticity of host '192.168.244.136 (192.168.244.136)' can't be 
established.
RSA key fingerprint is af:f2:1a:3a:2b:7c:c3:3b:6a:04:4f:37:bb:75:16:58.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.244.136' (RSA) to the list of known 
hosts.
root@192.168.244.136's password:
Linux debian 3.1.0-1-amd64 #1 SMP Tue Jan 10 05:01:58 UTC 2012 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Jan 19 15:06:22 2012 from login.cirrus.pdl.cmu.local
debian:~#
debian:~# uname -a
Linux debian 3.1.0-1-amd64 #1 SMP Tue Jan 10 05:01:58 UTC 2012 x86_64 
GNU/Linux
debian:~# cat /proc/cpuinfo
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 6
model           : 2
model name      : QEMU Virtual CPU version 0.14.0
stepping        : 3
cpu MHz         : 2193.593
cache size      : 512 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 4
wp              : yes
flags           : fpu pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov 
pat pse36 clflush mmx fxsr sse sse2 syscall nx lm up nopl pni cx16 
popcnt hypervisor lahf_lm svm abm sse4a
bogomips        : 4387.18
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

debian:~# echo "my new vm!"
my new vm
debian:~# halt

Broadcast message from root@debian (pts/0) (Wed Jan 25 02:01:43 2012):

The system is going down for system halt NOW!
debian:~# Connection to 192.168.244.136 closed by remote host.
Connection to 192.168.244.136 closed.

(XXXstroucki: the leaked stuff comes from lvm2 and can be suppressed by 
setting the environment value LVM_SUPPRESS_FD_WARNINGS. Current attitude 
is for lvm2 to no longer display the messages by default.)
(XXXstroucki: The lvremove output should be quieted.)

root@grml:/usr/local/tashi/bin# 2012-01-25 08:01:56,544 
[/usr/local/tashi/src/tashi/nodemanager/vmcontrol/qemu.py:INFO] Removing 
vmId 4370 because it is no longer running
2012-01-25 08:01:56,545 
[/usr/local/tashi/src/tashi/nodemanager/vmcontrol/qemu.py:INFO] Removing 
any scratch for wheezy
File descriptor 3 (socket:[18519]) leaked on lvremove invocation. Parent 
PID 4293: /usr/bin/python
File descriptor 5 (socket:[18526]) leaked on lvremove invocation. Parent 
PID 4293: /usr/bin/python
File descriptor 7 (pipe:[18747]) leaked on lvremove invocation. Parent 
PID 4293: /usr/bin/python
File descriptor 8 (/dev/pts/5 (deleted)) leaked on lvremove invocation. 
Parent PID 4293: /usr/bin/python
  Volume group "vgscratch" not found
  Skipping volume group vgscratch
2012-01-25 08:01:57,245 [./primitive:INFO] VM exited: wheezy

Verify the VM is no longer running:

root@grml:/usr/local/tashi/bin# ./tashi-client getinstances
 id hostId name user state disk memory cores
--------------------------------------------

Script done on Wed 25 Jan 2012 08:02:32 AM CET

Mime
View raw message