incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lu Heng <h...@anytimechinese.com>
Subject Re: Secondary storage is not working
Date Tue, 12 Jun 2012 12:09:53 GMT
Hi

I didn't see anything interesting here. Here is the most recent version(but
I have looked at the rotated vew ones, looks the same)

[13125] 2012-06-12 05:01:41.648419 #### VMOPS enter  get_rule_logs_for_vms
####
[13125] 2012-06-12 05:01:41.683571 #### VMOPS enter
 network_rules_for_rebooted_vm ####
[13125] 2012-06-12 05:01:41.683614 #### VMOPS enter  check_domid_changed
####
[13125] 2012-06-12 05:01:41.693652 #### VMOPS exit  check_domid_changed ####
[13125] 2012-06-12 05:01:41.693690 #### VMOPS exit
 network_rules_for_rebooted_vm ####
[13125] 2012-06-12 05:01:41.693732 #### VMOPS exit  get_rule_logs_for_vms
####
[13181] 2012-06-12 05:01:49.672472 #### VMOPS enter  gethostvmstats ####
[13181] 2012-06-12 05:01:49.680152 #### VMOPS exit  gethostvmstats ####
[13189] 2012-06-12 05:01:50.688317 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13189] 2012-06-12 05:01:50.696646 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13189] 2012-06-12 05:01:50.734480 lock: acquired
/var/lock/sm/iscsiadm/running
[13189] 2012-06-12 05:01:50.746575 lock: released
/var/lock/sm/iscsiadm/running
[13189] 2012-06-12 05:01:50.746687 lock: closed
/var/lock/sm/iscsiadm/running
[13189] 2012-06-12 05:01:50.747452 ['ls', '/sys/class/scsi_host', '-1',
'--color=never']
[13189] 2012-06-12 05:01:50.759330 SUCCESS
[13189] 2012-06-12 05:01:50.759950 ['11']
[13189] 2012-06-12 05:01:50.763759 PATHDICT: key 10.2.0.5:3260: {'path':
'/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260',
'ipaddr': '10.2.0.5', 'port': 3260}
[13189] 2012-06-12 05:01:50.763857 Path found:
/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260
[13189] 2012-06-12 05:01:50.777597 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13189] 2012-06-12 05:01:50.792979 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.815790 SUCCESS
[13189] 2012-06-12 05:01:50.816004 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.816106 LVMCache: will initialize now
[13189] 2012-06-12 05:01:50.816175 LVMCache: refreshing
[13189] 2012-06-12 05:01:50.816269 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.839279 SUCCESS
[13189] 2012-06-12 05:01:50.839672 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.839784 Entering _checkMetadataVolume
[13189] 2012-06-12 05:01:50.841193 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.841309 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13189] 2012-06-12 05:01:50.856727 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.878072 SUCCESS
[13189] 2012-06-12 05:01:50.878301 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.878406 LVMCache: will initialize now
[13189] 2012-06-12 05:01:50.878475 LVMCache: refreshing
[13189] 2012-06-12 05:01:50.878568 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.900543 SUCCESS
[13189] 2012-06-12 05:01:50.900936 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.901037 Entering _checkMetadataVolume
[13189] 2012-06-12 05:01:50.901347 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:50.901531 sr_scan {'sr_uuid':
'900fa90a-7ac7-eb3e-515d-ee1cd02c56b4', 'subtask_of':
'OpaqueRef:ebe4d090-bf89-d8d4-c200-591d95395fa5', 'args': [], 'host_ref':
'OpaqueRef:22f17b61-78fd-d1a0-ff65-703ad82a6edc', 'session_ref':
'OpaqueRef:d89486fe-07fc-182b-0ba9-f770211adfdc', 'device_config':
{'device':
'/dev/disk/by-id/scsi-14f504e46494c455243446d5854632d796750692d4d443439',
'SCSIid': '14f504e46494c455243446d5854632d796750692d4d443439', 'targetIQN':
'iqn.2006-01.com.openfiler:tsn.2912cb92f82d', 'SRmaster': 'true', 'target':
'10.2.0.5'}, 'command': 'sr_scan', 'sr_ref':
'OpaqueRef:75e09058-a764-2318-057e-1a17eb3ece81', 'local_cache_sr':
'f0c11a20-8883-ef1c-3290-0ab7fbc130fc'}
[13189] 2012-06-12 05:01:50.901698 LVHDSR.scan for
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13189] 2012-06-12 05:01:50.901922 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.943280 SUCCESS
[13189] 2012-06-12 05:01:50.944136 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13189] 2012-06-12 05:01:50.965564 SUCCESS
[13189] 2012-06-12 05:01:51.038455 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running, acquired: True
(exists: True)
[13189] 2012-06-12 05:01:51.038507 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13189] 2012-06-12 05:01:51.038537 Kicking GC
<13189> 2012-06-12 05:01:51.038575 === SR
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4: gc ===
<13223> 2012-06-12 05:01:51.040066 Will finish as PID [13224]
<13189> 2012-06-12 05:01:51.040723 New PID [13223]
[13189] 2012-06-12 05:01:51.040941 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13189] 2012-06-12 05:01:51.041280 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13189] 2012-06-12 05:01:51.047912 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13224] 2012-06-12 05:01:51.094792 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13224] 2012-06-12 05:01:51.112970 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr, acquired: True
(exists: True)
[13224] 2012-06-12 05:01:51.113013 LVMCache: refreshing
[13224] 2012-06-12 05:01:51.113050 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13224] 2012-06-12 05:01:51.136424 SUCCESS
[13224] 2012-06-12 05:01:51.136869 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13224] 2012-06-12 05:01:51.179332 SUCCESS
<13224> 2012-06-12 05:01:51.180168 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees):
        *0de05e6d[VHD](2.000G//540.000M|ao)
            fedba9ba[VHD](2.000G//2.012G|ao)
            5bca5a2b[VHD](2.000G//2.012G|n)
            bf63b940[VHD](2.000G//8.000M|n)

[13224] 2012-06-12 05:01:51.180354 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13224] 2012-06-12 05:01:51.180523 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13224] 2012-06-12 05:01:51.201612 SUCCESS
[13224] 2012-06-12 05:01:51.201856 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13224] 2012-06-12 05:01:51.223280 SUCCESS
<13224> 2012-06-12 05:01:51.238128 Got sm-config for
*0de05e6d[VHD](2.000G//540.000M|ao): {'vdi_type': 'vhd', 'vhd-blocks':
'eJz7//8AAwic/w9hAOkGEH2GAQLOP4DQJ6B8uPh/MPgHpj9A+Acg4mAd53+g6sNBMwIAgSMoSw=='}
<13224> 2012-06-12 05:01:51.238235 No work, exiting
<13224> 2012-06-12 05:01:51.238473 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): no changes
[13367] 2012-06-12 05:02:41.632069 #### VMOPS enter  get_rule_logs_for_vms
####
[13367] 2012-06-12 05:02:41.657769 #### VMOPS enter
 network_rules_for_rebooted_vm ####
[13367] 2012-06-12 05:02:41.657863 #### VMOPS enter  check_domid_changed
####
[13367] 2012-06-12 05:02:41.669337 #### VMOPS exit  check_domid_changed ####
[13367] 2012-06-12 05:02:41.669374 #### VMOPS exit
 network_rules_for_rebooted_vm ####
[13367] 2012-06-12 05:02:41.669416 #### VMOPS exit  get_rule_logs_for_vms
####
[13412] 2012-06-12 05:02:50.192681 #### VMOPS enter  gethostvmstats ####
[13412] 2012-06-12 05:02:50.200365 #### VMOPS exit  gethostvmstats ####
[13420] 2012-06-12 05:02:51.214194 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13420] 2012-06-12 05:02:51.222629 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13420] 2012-06-12 05:02:51.260176 lock: acquired
/var/lock/sm/iscsiadm/running
[13420] 2012-06-12 05:02:51.271762 lock: released
/var/lock/sm/iscsiadm/running
[13420] 2012-06-12 05:02:51.271872 lock: closed
/var/lock/sm/iscsiadm/running
[13420] 2012-06-12 05:02:51.272634 ['ls', '/sys/class/scsi_host', '-1',
'--color=never']
[13420] 2012-06-12 05:02:51.284386 SUCCESS
[13420] 2012-06-12 05:02:51.285007 ['11']
[13420] 2012-06-12 05:02:51.288837 PATHDICT: key 10.2.0.5:3260: {'path':
'/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260',
'ipaddr': '10.2.0.5', 'port': 3260}
[13420] 2012-06-12 05:02:51.288936 Path found:
/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260
[13420] 2012-06-12 05:02:51.302907 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13420] 2012-06-12 05:02:51.318194 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.339945 SUCCESS
[13420] 2012-06-12 05:02:51.340163 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.340265 LVMCache: will initialize now
[13420] 2012-06-12 05:02:51.340334 LVMCache: refreshing
[13420] 2012-06-12 05:02:51.340431 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.363255 SUCCESS
[13420] 2012-06-12 05:02:51.363653 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.363758 Entering _checkMetadataVolume
[13420] 2012-06-12 05:02:51.365197 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.365314 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13420] 2012-06-12 05:02:51.380298 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.401257 SUCCESS
[13420] 2012-06-12 05:02:51.401471 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.401572 LVMCache: will initialize now
[13420] 2012-06-12 05:02:51.401641 LVMCache: refreshing
[13420] 2012-06-12 05:02:51.401734 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.424356 SUCCESS
[13420] 2012-06-12 05:02:51.424750 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.424856 Entering _checkMetadataVolume
[13420] 2012-06-12 05:02:51.425167 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.425353 sr_scan {'sr_uuid':
'900fa90a-7ac7-eb3e-515d-ee1cd02c56b4', 'subtask_of':
'OpaqueRef:672a5cf7-d938-c6bd-001f-7b59d4a69204', 'args': [], 'host_ref':
'OpaqueRef:22f17b61-78fd-d1a0-ff65-703ad82a6edc', 'session_ref':
'OpaqueRef:af2690fc-6539-daf2-ff58-30fe2f9a0371', 'device_config':
{'device':
'/dev/disk/by-id/scsi-14f504e46494c455243446d5854632d796750692d4d443439',
'SCSIid': '14f504e46494c455243446d5854632d796750692d4d443439', 'targetIQN':
'iqn.2006-01.com.openfiler:tsn.2912cb92f82d', 'SRmaster': 'true', 'target':
'10.2.0.5'}, 'command': 'sr_scan', 'sr_ref':
'OpaqueRef:75e09058-a764-2318-057e-1a17eb3ece81', 'local_cache_sr':
'f0c11a20-8883-ef1c-3290-0ab7fbc130fc'}
[13420] 2012-06-12 05:02:51.425503 LVHDSR.scan for
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13420] 2012-06-12 05:02:51.425726 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.468757 SUCCESS
[13420] 2012-06-12 05:02:51.469610 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13420] 2012-06-12 05:02:51.491025 SUCCESS
[13420] 2012-06-12 05:02:51.564940 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running, acquired: True
(exists: True)
[13420] 2012-06-12 05:02:51.564993 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13420] 2012-06-12 05:02:51.565033 Kicking GC
<13420> 2012-06-12 05:02:51.565073 === SR
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4: gc ===
<13454> 2012-06-12 05:02:51.566603 Will finish as PID [13455]
<13420> 2012-06-12 05:02:51.567259 New PID [13454]
[13420] 2012-06-12 05:02:51.567476 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13420] 2012-06-12 05:02:51.567812 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13420] 2012-06-12 05:02:51.574404 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13455] 2012-06-12 05:02:51.621214 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13455] 2012-06-12 05:02:51.639586 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr, acquired: True
(exists: True)
[13455] 2012-06-12 05:02:51.639629 LVMCache: refreshing
[13455] 2012-06-12 05:02:51.639667 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13455] 2012-06-12 05:02:51.662316 SUCCESS
[13455] 2012-06-12 05:02:51.662762 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13455] 2012-06-12 05:02:51.704936 SUCCESS
<13455> 2012-06-12 05:02:51.705771 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees):
        *0de05e6d[VHD](2.000G//540.000M|ao)
            fedba9ba[VHD](2.000G//2.012G|ao)
            5bca5a2b[VHD](2.000G//2.012G|n)
            bf63b940[VHD](2.000G//8.000M|n)

[13455] 2012-06-12 05:02:51.705959 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13455] 2012-06-12 05:02:51.706127 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13455] 2012-06-12 05:02:51.727751 SUCCESS
[13455] 2012-06-12 05:02:51.727992 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13455] 2012-06-12 05:02:51.748946 SUCCESS
<13455> 2012-06-12 05:02:51.761594 Got sm-config for
*0de05e6d[VHD](2.000G//540.000M|ao): {'vdi_type': 'vhd', 'vhd-blocks':
'eJz7//8AAwic/w9hAOkGEH2GAQLOP4DQJ6B8uPh/MPgHpj9A+Acg4mAd53+g6sNBMwIAgSMoSw=='}
<13455> 2012-06-12 05:02:51.761638 No work, exiting
<13455> 2012-06-12 05:02:51.761737 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): no changes
[13614] 2012-06-12 05:03:41.631406 #### VMOPS enter  get_rule_logs_for_vms
####
[13614] 2012-06-12 05:03:41.656623 #### VMOPS enter
 network_rules_for_rebooted_vm ####
[13614] 2012-06-12 05:03:41.656718 #### VMOPS enter  check_domid_changed
####
[13614] 2012-06-12 05:03:41.668513 #### VMOPS exit  check_domid_changed ####
[13614] 2012-06-12 05:03:41.668551 #### VMOPS exit
 network_rules_for_rebooted_vm ####
[13614] 2012-06-12 05:03:41.668593 #### VMOPS exit  get_rule_logs_for_vms
####
[13659] 2012-06-12 05:03:50.674948 #### VMOPS enter  gethostvmstats ####
[13659] 2012-06-12 05:03:50.682474 #### VMOPS exit  gethostvmstats ####
[13669] 2012-06-12 05:03:51.816879 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13669] 2012-06-12 05:03:51.825230 Setting LVM_DEVICE to
/dev/disk/by-scsid/14f504e46494c455243446d5854632d796750692d4d443439
[13669] 2012-06-12 05:03:51.861581 lock: acquired
/var/lock/sm/iscsiadm/running
[13669] 2012-06-12 05:03:51.873663 lock: released
/var/lock/sm/iscsiadm/running
[13669] 2012-06-12 05:03:51.873776 lock: closed
/var/lock/sm/iscsiadm/running
[13669] 2012-06-12 05:03:51.874533 ['ls', '/sys/class/scsi_host', '-1',
'--color=never']
[13669] 2012-06-12 05:03:51.884299 SUCCESS
[13669] 2012-06-12 05:03:51.884918 ['11']
[13669] 2012-06-12 05:03:51.888744 PATHDICT: key 10.2.0.5:3260: {'path':
'/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260',
'ipaddr': '10.2.0.5', 'port': 3260}
[13669] 2012-06-12 05:03:51.888843 Path found:
/dev/iscsi/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/10.2.0.5:3260
[13669] 2012-06-12 05:03:51.902943 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13669] 2012-06-12 05:03:51.918180 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:51.940914 SUCCESS
[13669] 2012-06-12 05:03:51.941132 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:51.941234 LVMCache: will initialize now
[13669] 2012-06-12 05:03:51.941304 LVMCache: refreshing
[13669] 2012-06-12 05:03:51.941401 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:51.964523 SUCCESS
[13669] 2012-06-12 05:03:51.964912 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:51.965019 Entering _checkMetadataVolume
[13669] 2012-06-12 05:03:51.966443 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:51.966560 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13669] 2012-06-12 05:03:51.982203 ['/usr/sbin/vgs',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:52.003569 SUCCESS
[13669] 2012-06-12 05:03:52.003665 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:52.003711 LVMCache: will initialize now
[13669] 2012-06-12 05:03:52.003741 LVMCache: refreshing
[13669] 2012-06-12 05:03:52.003784 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:52.025826 SUCCESS
[13669] 2012-06-12 05:03:52.026232 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:52.026336 Entering _checkMetadataVolume
[13669] 2012-06-12 05:03:52.026648 lock: acquired
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:52.026834 sr_scan {'sr_uuid':
'900fa90a-7ac7-eb3e-515d-ee1cd02c56b4', 'subtask_of':
'OpaqueRef:70e76ca3-6e80-c865-98bb-9dbfa362379a', 'args': [], 'host_ref':
'OpaqueRef:22f17b61-78fd-d1a0-ff65-703ad82a6edc', 'session_ref':
'OpaqueRef:2aa1c42e-3b05-c271-bf60-57428a855e46', 'device_config':
{'device':
'/dev/disk/by-id/scsi-14f504e46494c455243446d5854632d796750692d4d443439',
'SCSIid': '14f504e46494c455243446d5854632d796750692d4d443439', 'targetIQN':
'iqn.2006-01.com.openfiler:tsn.2912cb92f82d', 'SRmaster': 'true', 'target':
'10.2.0.5'}, 'command': 'sr_scan', 'sr_ref':
'OpaqueRef:75e09058-a764-2318-057e-1a17eb3ece81', 'local_cache_sr':
'f0c11a20-8883-ef1c-3290-0ab7fbc130fc'}
[13669] 2012-06-12 05:03:52.026984 LVHDSR.scan for
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13669] 2012-06-12 05:03:52.027207 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:52.070068 SUCCESS
[13669] 2012-06-12 05:03:52.070933 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13669] 2012-06-12 05:03:52.092699 SUCCESS
[13669] 2012-06-12 05:03:52.160000 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running, acquired: True
(exists: True)
[13669] 2012-06-12 05:03:52.160052 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13669] 2012-06-12 05:03:52.160082 Kicking GC
<13669> 2012-06-12 05:03:52.160121 === SR
900fa90a-7ac7-eb3e-515d-ee1cd02c56b4: gc ===
<13703> 2012-06-12 05:03:52.161610 Will finish as PID [13704]
<13669> 2012-06-12 05:03:52.162260 New PID [13703]
[13669] 2012-06-12 05:03:52.162484 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/running
[13669] 2012-06-12 05:03:52.162823 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13669] 2012-06-12 05:03:52.169460 lock: closed
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13704] 2012-06-12 05:03:52.221888 LVMCache created for
VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4
[13704] 2012-06-12 05:03:52.256927 lock: tried lock
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr, acquired: True
(exists: True)
[13704] 2012-06-12 05:03:52.257025 LVMCache: refreshing
[13704] 2012-06-12 05:03:52.257133 ['/usr/sbin/lvs', '--noheadings',
'--units', 'b', '-o', '+lv_tags',
'/dev/VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13704] 2012-06-12 05:03:52.282878 SUCCESS
[13704] 2012-06-12 05:03:52.283329 ['/usr/sbin/vhd-util', 'scan', '-f',
'-c', '-m', 'VHD-*', '-l',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13704] 2012-06-12 05:03:52.332991 SUCCESS
<13704> 2012-06-12 05:03:52.333830 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees):
        *0de05e6d[VHD](2.000G//540.000M|ao)
            fedba9ba[VHD](2.000G//2.012G|ao)
            5bca5a2b[VHD](2.000G//2.012G|n)
            bf63b940[VHD](2.000G//8.000M|n)

[13704] 2012-06-12 05:03:52.334019 lock: released
/var/lock/sm/900fa90a-7ac7-eb3e-515d-ee1cd02c56b4/sr
[13704] 2012-06-12 05:03:52.334186 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13704] 2012-06-12 05:03:52.354072 SUCCESS
[13704] 2012-06-12 05:03:52.354315 ['/usr/sbin/vgs', '--noheadings',
'--nosuffix', '--units', 'b',
'VG_XenStorage-900fa90a-7ac7-eb3e-515d-ee1cd02c56b4']
[13704] 2012-06-12 05:03:52.376106 SUCCESS
<13704> 2012-06-12 05:03:52.393268 Got sm-config for
*0de05e6d[VHD](2.000G//540.000M|ao): {'vdi_type': 'vhd', 'vhd-blocks':
'eJz7//8AAwic/w9hAOkGEH2GAQLOP4DQJ6B8uPh/MPgHpj9A+Acg4mAd53+g6sNBMwIAgSMoSw=='}
<13704> 2012-06-12 05:03:52.393377 No work, exiting
<13704> 2012-06-12 05:03:52.393603 SR 900f
('f7d6960c-84a6-332f-9068-27ef0f216b31') (4 VDIs in 1 VHD trees): no changes

On Tue, Jun 12, 2012 at 1:21 PM, Nitin Mehta <Nitin.Mehta@citrix.com> wrote:

> I am assuming you are using Xenserver. Log in to the hypervisor. “cd
> /var/log/”. Here you can find the logs. These logs get rotated so be sure
> to search in all of them.****
>
> I am asking to you to look here because the operation
> PrimaryStorageDownloadCommand is failing which downloads the template from
> secondary to primary storage while creating system vms.****
>
> ** **
>
> Thanks,****
>
> -Nitin****
>
> ** **
>
> *From:* Lu Heng [mailto:h.lu@anytimechinese.com]
> *Sent:* Tuesday, June 12, 2012 3:27 PM
> *To:* Nitin Mehta
> *Cc:* cloudstack-dev@incubator.apache.org
>
> *Subject:* Re: Secondary storage is not working****
>
> ** **
>
> Hi****
>
> On Tue, Jun 12, 2012 at 6:02 AM, Nitin Mehta <Nitin.Mehta@citrix.com>
> wrote:****
>
> Lu - With the information provided it seems that the system vm template
> and secondary storage have been configured correctly.
> Let's circle back to the error in the logs then. Can you try and manually
> mount the entire path (1.1.128.2:/data/template/tmpl/1/1) on the
> xenserver ?****
>
> ** **
>
> I tried, it works. ****
>
> com.cloud.utils.exception.CloudRuntimeException: can't mount
> 1.1.128.2:/data/template/tmpl/1/1****
>
> to /var/run/cloud_mount/f2c6f574-35f4-4552-80a8-0052b7b70bde ?
>
> If this works then let's look into the logs of this failure which should
> be available on SMlog on the hypervisor.  Search for the mounting error and
> see if you can find something there.****
>
> SMlog, how can I find it? ****
>
>
> Thanks,
> -Nitin
>
> -----Original Message-----
> From: Lu Heng [mailto:h.lu@anytimechinese.com]****
>
> Sent: Tuesday, June 12, 2012 9:19 AM
> To: cloudstack-dev@incubator.apache.org
> Cc: Nitin Mehta
> Subject: Re: Secondary storage is not working
>
> Hi****
>
> some more information:
>
> Server IP:1.1.1.1(replaced with example ip, but the path is the same)
> path:/data
>
> preparation of the system template:
> /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmplt
> -m /data -u
> http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
> -h<http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2%0d%0a-h>xenserver
-F
> --2012-06-11 23:44:07--
> http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2
> Resolving download.cloud.com... 207.171.185.201 Connecting to
> download.cloud.com|207.171.185.201|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 140616708 (134M) [binary/octet-stream] Saving to:
> `/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
>
>
> 100%[==================================================================================================================================>]
> 140,616,708 42.4M/s   in 4.7s
>
> 2012-06-11 23:44:12 (28.3 MB/s) -
>
> `/usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd'
> saved [140616708/140616708]
>
> Uncompressing to
>
> /usr/lib64/cloud/agent/scripts/storage/secondary/8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd.tmp
> (type bz2)...could take a long time
> Moving to
> /data/template/tmpl/1/1///8ab03e88-d05e-4a86-bb00-cca4ad60f4f6.vhd...could
> take a while
> Successfully installed system VM template  to /data/template/tmpl/1/1/
>
> /etc/exports:
> /data *(rw,async,no_root_squash)
>
> Please let me know if you need more information.
> On Tue, Jun 12, 2012 at 5:30 AM, Lu Heng <h.lu@anytimechinese.com> wrote:
> > Hi Nitin:
> >
> > Thanks for replying.
> >
> > See my reply.
> >
> > On Tue, Jun 12, 2012 at 5:09 AM, Nitin Mehta <Nitin.Mehta@citrix.com>
> wrote:
> >>
> >> Lu,
> >>
> >> Can you paste the snippet of logs where you see the error? Can you
> please elaborate on what you mean by secondary storage setup is not working.
> >> In my knowledge, MS doesn't validate the secondary storage path and
> does very basic tests while adding it. Typically I have seen the following
> issues users commit while adding secondary storage.
> >> Please check if your issue falls under one of them.
> >>
> >> 1.      Users skip the preparation of the storage (e.g. not seeding
> >> the System VM to the NFS export or seeding the wrong version for the
> >> hypervisor)
> >
> > /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmp
> > lt -m /mnt/secondary -u
> > http://download.cloud.com/templates/acton/acton-systemvm-
> > 02062012.vhd.bz2 -h xenserver -F
> > is proformed in the management server, in which
> > /data/template/tmpl/1/1 was created, and a vhd file together with a
> > template.properties was under that folder.
> > three things I saw:
> > 1. under template section of the cloudstack, I saw SystemVM Template
> > (XenServer)   usdatacenter1   XenServer, and if I open it, I saw
> > "download complete"
> > However, seems it have nothing to do with the real storage, as if I
> > delete the secondary storage, it is still there.
> > 2. in the installation guide it says it tooks up to 30 mins to
> > download that template, however, I saw "successful installation" after
> > about 30 seconds.
> > 3. seems cloud stack does not check the secondary storage, i tried to
> > add random IP address with random path, it still shows "enabled".
> >
> >>
> >> 2.      NFS v4 is used which causes errors during Secondary Storage
> >> preparation (Solution: mount the storage and force NFS v3 before
> >> preparing the Storage: mount -v -t nfs -o vers=3
> >> servername:/nfs/share /mnt/secondary)
> > output from the management server:
> > Server rpc stats:
> > calls      badcalls   badauth    badclnt    xdrcall
> > 3803       0          0          0          0
> >
> > Server nfs v3:
> > null         getattr      setattr      lookup       access
> > readlink
> > 3232     85% 12        0% 0         0% 0         0% 5         0% 0
>
> > 0% read         write        create       mkdir        symlink
> > mknod
> > 545      14% 0         0% 0         0% 0         0% 0         0% 0
>
> > 0% remove       rmdir        rename       link         readdir
> > readdirplus
> > 0         0% 0         0% 0         0% 0         0% 0         0% 3
>
> > 0% fsstat       fsinfo       pathconf     commit
> > 0         0% 4         0% 0         0% 0         0%
> >
> >> 3.      The /etc/export file on the NFS does not contain
> 'no_root_squash'
> > It does.
> >> 4.      The /etc/export file on the NFS blocks the host/Management
> >> Network CIDR
> > the NFS is on the management server.
> >> 5.      Users enter wrong NFS share path (no validation by
> >> CloudStack)
> > it is correct as I have successfully mounted it from Host sever.
> >>
> >> Thanks,
> >> -Nitin
> >>
> >> -----Original Message-----
> >> From: Lu Heng [mailto:h.lu@anytimechinese.com]
> >> Sent: Tuesday, June 12, 2012 7:59 AM
> >> To: cloudstack-dev@incubator.apache.org
> >> Subject: Secondary storage is not working
> >>
> >> Hi
> >>
> >>  I have secondary storage setup at management server's /data path, and
> while I add it to the zone, it is not working(with information
> server:<management-server-IP> path:/data), however,I can successfully mount
> secondary storage in the HV server, any one have idea what is wrong?
> >>
> >>
> >> --
> >> --
> >> Kind regards.
> >> Lu
> >>
> >> This transmission is intended solely for the addressee(s) shown above.
> >> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use of
> this transmission or its contents by persons other than the intended
> addressee(s) is strictly prohibited. If you have received this transmission
> in error, please notify this office immediately and e-mail the original at
> the sender's address above by replying to this message and including the
> text of the transmission received.
> >
> >
> >
> >
> > --
> > --
> > Kind regards.
> > Lu
> >
> > This transmission is intended solely for the addressee(s) shown above.
> > It may contain information that is privileged, confidential or
> > otherwise protected from disclosure. Any review, dissemination or use
> > of this transmission or its contents by persons other than the
> > intended addressee(s) is strictly prohibited. If you have received
> > this transmission in error, please notify this office immediately and
> > e-mail the original at the sender's address above by replying to this
> > message and including the text of the transmission received.
>
>
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or otherwise
> protected from disclosure. Any review, dissemination or use of this
> transmission or its contents by persons other than the intended
> addressee(s) is strictly prohibited. If you have received this transmission
> in error, please notify this office immediately and e-mail the original at
> the sender's address above by replying to this message and including the
> text of the transmission received.****
>
>
>
> ****
>
> ** **
>
> --
> --
> Kind regards.
> Lu
>
> This transmission is intended solely for the addressee(s) shown above.
> It may contain information that is privileged, confidential or
> otherwise protected from disclosure. Any review, dissemination or use
> of this transmission or its contents by persons other than the
> intended addressee(s) is strictly prohibited. If you have received
> this transmission in error, please notify this office immediately and
> e-mail the original at the sender's address above by replying to this
> message and including the text of the transmission received.****
>



-- 
--
Kind regards.
Lu

This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or
otherwise protected from disclosure. Any review, dissemination or use
of this transmission or its contents by persons other than the
intended addressee(s) is strictly prohibited. If you have received
this transmission in error, please notify this office immediately and
e-mail the original at the sender's address above by replying to this
message and including the text of the transmission received.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message