cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Tutkowski <mike.tutkow...@solidfire.com>
Subject Re: Managed storage with KVM
Date Thu, 26 Sep 2013 00:07:36 GMT
It's been a bit rough getting this up and running, but at least I've been
learning about how CloudStack works on KVM, so that's really good.


On Wed, Sep 25, 2013 at 6:03 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:

> I mean switch over to 4.2 from master. :)
>
>
> On Wed, Sep 25, 2013 at 6:03 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> I can switch my branch over to master. I'm afraid master is not working
>> with Libvirt on Ubuntu, as well.
>>
>>
>> On Wed, Sep 25, 2013 at 5:55 PM, Marcus Sorensen <shadowsor@gmail.com>wrote:
>>
>>> It's harder still that you're trying to use master. I know 4.2 works
>>> on ubuntu, but master is a minefield sometimes. Maybe that's not the
>>> problem, but I do see emails going back and forth about libvirt/jna
>>> versions, just need to read them in detail.
>>>
>>>  It's a shame that you haven't gotten a working config up yet prior to
>>> development work (say a 4.2 that we know works), because we don't have
>>> any clues as to whether it's your setup or master.
>>>
>>> On Wed, Sep 25, 2013 at 5:49 PM, Marcus Sorensen <shadowsor@gmail.com>
>>> wrote:
>>> > ok, just a guess. I'm assuming it's still this:
>>> >
>>> > Caused by: java.lang.NoSuchMethodError: com.sun.jna.Native.free(J)V
>>> >
>>> > On Wed, Sep 25, 2013 at 5:48 PM, Mike Tutkowski
>>> > <mike.tutkowski@solidfire.com> wrote:
>>> >> mtutkowski@ubuntu:~$ sudo apt-get install libjna-java
>>> >> Reading package lists... Done
>>> >> Building dependency tree
>>> >> Reading state information... Done
>>> >> libjna-java is already the newest version.
>>> >> libjna-java set to manually installed.
>>> >> 0 upgraded, 0 newly installed, 0 to remove and 468 not upgraded.
>>> >>
>>> >>
>>> >> On Wed, Sep 25, 2013 at 5:46 PM, Mike Tutkowski <
>>> >> mike.tutkowski@solidfire.com> wrote:
>>> >>
>>> >>> Was there a step in the docs I may have missed where I was to install
>>> >>> them? I don't recall installing them, but there are several steps
>>> and I
>>> >>> might have forgotten that I did install them, too.
>>> >>>
>>> >>> I can check.
>>> >>>
>>> >>>
>>> >>> On Wed, Sep 25, 2013 at 5:44 PM, Marcus Sorensen <
>>> shadowsor@gmail.com>wrote:
>>> >>>
>>> >>>> are you missing the jna packages?
>>> >>>>
>>> >>>> On Wed, Sep 25, 2013 at 5:40 PM, Mike Tutkowski
>>> >>>> <mike.tutkowski@solidfire.com> wrote:
>>> >>>> > I basically just leveraged the code you provided to redirect
the
>>> output
>>> >>>> on
>>> >>>> > Ubuntu.
>>> >>>> >
>>> >>>> > Here is the standard err:
>>> >>>> >
>>> >>>> > log4j:WARN No appenders could be found for logger
>>> >>>> > (org.apache.commons.httpclient.params.DefaultHttpParams).
>>> >>>> > log4j:WARN Please initialize the log4j system properly.
>>> >>>> > log4j:WARN See
>>> http://logging.apache.org/log4j/1.2/faq.html#noconfigfor
>>> >>>> > more info.
>>> >>>> > java.lang.reflect.InvocationTargetException
>>> >>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
>>> >>>> > at
>>> >>>> >
>>> >>>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> >>>> > at
>>> >>>> >
>>> >>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> >>>> > at java.lang.reflect.Method.invoke(Method.java:606)
>>> >>>> > at
>>> >>>> >
>>> >>>>
>>> org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
>>> >>>> > Caused by: java.lang.NoSuchMethodError:
>>> com.sun.jna.Native.free(J)V
>>> >>>> > at org.libvirt.Library.free(Unknown Source)
>>> >>>> > at org.libvirt.Connect.getCapabilities(Unknown Source)
>>> >>>> > at
>>> >>>> >
>>> >>>>
>>> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.IsHVMEnabled(LibvirtComputingResource.java:4524)
>>> >>>> > at
>>> >>>> >
>>> >>>>
>>> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.configure(LibvirtComputingResource.java:753)
>>> >>>> > at com.cloud.agent.Agent.<init>(Agent.java:168)
>>> >>>> > at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:439)
>>> >>>> > at
>>> >>>>
>>> com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:386)
>>> >>>> > at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:361)
>>> >>>> > at com.cloud.agent.AgentShell.start(AgentShell.java:473)
>>> >>>> > ... 5 more
>>> >>>> > Cannot start daemon
>>> >>>> > Service exit with a return value of 5
>>> >>>> >
>>> >>>> >
>>> >>>> > On Wed, Sep 25, 2013 at 5:07 PM, Mike Tutkowski <
>>> >>>> > mike.tutkowski@solidfire.com> wrote:
>>> >>>> >
>>> >>>> >> Sounds good.
>>> >>>> >>
>>> >>>> >> Thanks, Marcus! :)
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> On Wed, Sep 25, 2013 at 5:04 PM, Marcus Sorensen <
>>> shadowsor@gmail.com
>>> >>>> >wrote:
>>> >>>> >>
>>> >>>> >>> Ok, so the next step is to track that stdout and
see if you can
>>> see
>>> >>>> >>> what jsvc complains about when it fails to start
up the service.
>>> >>>> >>>
>>> >>>> >>> On Wed, Sep 25, 2013 at 4:56 PM, Mike Tutkowski
>>> >>>> >>> <mike.tutkowski@solidfire.com> wrote:
>>> >>>> >>> > These also look good:
>>> >>>> >>> >
>>> >>>> >>> > mtutkowski@ubuntu:/etc/cloudstack/agent$ uname
-m
>>> >>>> >>> > x86_64
>>> >>>> >>> > mtutkowski@ubuntu:/etc/cloudstack/agent$ virsh
-c
>>> qemu:///system
>>> >>>> list
>>> >>>> >>> >  Id Name                 State
>>> >>>> >>> > ----------------------------------
>>> >>>> >>> >
>>> >>>> >>> > mtutkowski@ubuntu:/etc/cloudstack/agent$ sudo
ls -la
>>> >>>> >>> > /var/run/libvirt/libvirt-sock
>>> >>>> >>> > srwxrwx--- 1 root libvirtd 0 Sep 25 16:05
>>> >>>> /var/run/libvirt/libvirt-sock
>>> >>>> >>> > mtutkowski@ubuntu:/etc/cloudstack/agent$ ls
-l /dev/kvm
>>> >>>> >>> > crw-rw----+ 1 root kvm 10, 232 Sep 25 15:22
/dev/kvm
>>> >>>> >>> >
>>> >>>> >>> >
>>> >>>> >>> >
>>> >>>> >>> > On Wed, Sep 25, 2013 at 4:53 PM, Mike Tutkowski
<
>>> >>>> >>> > mike.tutkowski@solidfire.com> wrote:
>>> >>>> >>> >
>>> >>>> >>> >> This is my new agent.properties file (with
comments
>>> removed...looks
>>> >>>> >>> >> decent):
>>> >>>> >>> >>
>>> >>>> >>> >> guid=6b4aa1c2-2ac9-3c60-aabe-704aed40c684
>>> >>>> >>> >>
>>> resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource
>>> >>>> >>> >> workers=5
>>> >>>> >>> >> host=192.168.233.1
>>> >>>> >>> >> port=8250
>>> >>>> >>> >> cluster=1
>>> >>>> >>> >> pod=1
>>> >>>> >>> >> zone=1
>>> >>>> >>> >> local.storage.uuid=aced86a2-2dd6-450a-93e5-1bc0ec3c73be
>>> >>>> >>> >> private.network.device=cloudbr0
>>> >>>> >>> >> public.network.device=cloudbr0
>>> >>>> >>> >> guest.network.device=cloudbr0
>>> >>>> >>> >>
>>> >>>> >>> >> Yeah, I was always writing stuff out using
the logger. I
>>> should
>>> >>>> look
>>> >>>> >>> into
>>> >>>> >>> >> redirecting stdout and stderr.
>>> >>>> >>> >>
>>> >>>> >>> >> Here were my steps to start and check
the process status:
>>> >>>> >>> >>
>>> >>>> >>> >> mtutkowski@ubuntu:/etc/cloudstack/agent$
sudo
>>> /usr/sbin/service
>>> >>>> >>> >> cloudstack-agent start
>>> >>>> >>> >>  * Starting CloudStack Agent cloudstack-agent
>>> >>>> >>> >>                                      
               [ OK ]
>>> >>>> >>> >> mtutkowski@ubuntu:/etc/cloudstack/agent$
sudo ps -ef | grep
>>> jsvc
>>> >>>> >>> >> 1000      4605  3725  0 16:47 pts/1  
 00:00:00 grep
>>> --color=auto
>>> >>>> jsvc
>>> >>>> >>> >>
>>> >>>> >>> >> Also, this might be of interest:
>>> >>>> >>> >>
>>> >>>> >>> >> mtutkowski@ubuntu:/etc/cloudstack/agent$
lsmod | grep kvm
>>> >>>> >>> >> kvm_intel             137721  0
>>> >>>> >>> >> kvm                   415549  1 kvm_intel
>>> >>>> >>> >>
>>> >>>> >>> >> mtutkowski@ubuntu:/etc/cloudstack/agent$
egrep -c
>>> '(vmx|svm)'
>>> >>>> >>> >> /proc/cpuinfo
>>> >>>> >>> >> 1
>>> >>>> >>> >>
>>> >>>> >>> >> mtutkowski@ubuntu:/etc/cloudstack/agent$
kvm-ok
>>> >>>> >>> >> INFO: /dev/kvm exists
>>> >>>> >>> >> KVM acceleration can be used
>>> >>>> >>> >>
>>> >>>> >>> >> mtutkowski@ubuntu:/etc/cloudstack/agent$
egrep -c ' lm '
>>> >>>> /proc/cpuinfo
>>> >>>> >>> >> 1
>>> >>>> >>> >>
>>> >>>> >>> >> On Wed, Sep 25, 2013 at 4:39 PM, Marcus
Sorensen <
>>> >>>> shadowsor@gmail.com
>>> >>>> >>> >wrote:
>>> >>>> >>> >>
>>> >>>> >>> >>> So you:
>>> >>>> >>> >>>
>>> >>>> >>> >>> 1. run that command
>>> >>>> >>> >>> 2. get a brand new agent.properties
as a result
>>> >>>> >>> >>> 3. start the service
>>> >>>> >>> >>>
>>> >>>> >>> >>> but you don't see it in the process
table?
>>> >>>> >>> >>>
>>> >>>> >>> >>> The agent's STDOUT doesn't go to the
agent log, only log4j
>>> stuff.
>>> >>>> So
>>> >>>> >>> >>> if there were an error not printed
via logger you'd not see
>>> it.
>>> >>>>  I'm
>>> >>>> >>> >>> not as familiar with the debian/ubuntu
stuff off the top of
>>> my
>>> >>>> head,
>>> >>>> >>> >>> but in /etc/init.d/cloudstack-agent
on CentOS we do:
>>> >>>> >>> >>>
>>> >>>> >>> >>> start() {
>>> >>>> >>> >>>     echo -n $"Starting $PROGNAME:
"
>>> >>>> >>> >>>     if hostname --fqdn >/dev/null
2>&1 ; then
>>> >>>> >>> >>>         $JSVC -cp "$CLASSPATH" -pidfile
"$PIDFILE" \
>>> >>>> >>> >>>             -errfile $LOGDIR/cloudstack-agent.err
-outfile
>>> >>>> >>> >>> $LOGDIR/cloudstack-agent.out $CLASS
>>> >>>> >>> >>>         RETVAL=$?
>>> >>>> >>> >>>         echo
>>> >>>> >>> >>>     else
>>> >>>> >>> >>>
>>> >>>> >>> >>>
>>> >>>> >>> >>> Which sends STDOUT to cloudstack-agent.out
and errors to
>>> >>>> >>> >>> cloudstack-agent.err. You can look
to see what Ubuntu does.
>>> >>>> >>> >>>
>>> >>>> >>> >>> Out of curiosity, what do you get
when you do 'lsmod | grep
>>> kvm'
>>> >>>> ? I
>>> >>>> >>> >>> know you didn't end up using it, but
the devcloud-kvm
>>> >>>> instructions for
>>> >>>> >>> >>> vmware fusion tell you to ensure that
your guest has
>>> hardware
>>> >>>> >>> >>> virtualization passthrough enabled,
I'm wondering if it
>>> isn't.
>>> >>>> >>> >>>
>>> >>>> >>> >>> On Wed, Sep 25, 2013 at 4:11 PM, Mike
Tutkowski
>>> >>>> >>> >>> <mike.tutkowski@solidfire.com>
wrote:
>>> >>>> >>> >>> > These results look good:
>>> >>>> >>> >>> >
>>> >>>> >>> >>> > mtutkowski@ubuntu:~$ sudo cloudstack-setup-agent
-m
>>> >>>> 192.168.233.1
>>> >>>> >>> -z 1
>>> >>>> >>> >>> -p 1
>>> >>>> >>> >>> > -c 1 -g 6b4aa1c2-2ac9-3c60-aabe-704aed40c684
-a
>>> >>>> --pubNic=cloudbr0
>>> >>>> >>> >>> > --prvNic=cloudbr0 --guestNic=cloudbr0
>>> >>>> >>> >>> > Starting to configure your system:
>>> >>>> >>> >>> > Configure Apparmor ...      
 [OK]
>>> >>>> >>> >>> > Configure Network ...       
 [OK]
>>> >>>> >>> >>> > Configure Libvirt ...       
 [OK]
>>> >>>> >>> >>> > Configure Firewall ...      
 [OK]
>>> >>>> >>> >>> > Configure Nfs ...           
 [OK]
>>> >>>> >>> >>> > Configure cloudAgent ...    
 [OK]
>>> >>>> >>> >>> > CloudStack Agent setup is done!
>>> >>>> >>> >>> >
>>> >>>> >>> >>> > However, these results are the
same:
>>> >>>> >>> >>> >
>>> >>>> >>> >>> > mtutkowski@ubuntu:~$ ps -ef |
grep jsvc
>>> >>>> >>> >>> > 1000      4314  3725  0 16:10
pts/1    00:00:00 grep
>>> >>>> --color=auto
>>> >>>> >>> jsvc
>>> >>>> >>> >>> >
>>> >>>> >>> >>> >
>>> >>>> >>> >>> > On Wed, Sep 25, 2013 at 3:48
PM, Mike Tutkowski <
>>> >>>> >>> >>> > mike.tutkowski@solidfire.com>
wrote:
>>> >>>> >>> >>> >
>>> >>>> >>> >>> >> This appears to be the offending
method:
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>     public String parseCapabilitiesXML(String
capXML) {
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>         if (!_initialized)
{
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>             return null;
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>         }
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>         try {
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>             _sp.parse(new
InputSource(new
>>> >>>> StringReader(capXML)),
>>> >>>> >>> this);
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>             return _capXML.toString();
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>         } catch (SAXException
se) {
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>             s_logger.warn(se.getMessage());
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>         } catch (IOException
ie) {
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>             s_logger.error(ie.getMessage());
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>         }
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>         return null;
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>     }
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >> The logging I do from this
method (not shown above),
>>> however,
>>> >>>> >>> doesn't
>>> >>>> >>> >>> seem
>>> >>>> >>> >>> >> to end up in agent.log. Not
sure why that is.
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >> We invoke this method and
I log we're in this method as
>>> the
>>> >>>> first
>>> >>>> >>> >>> thing I
>>> >>>> >>> >>> >> do, but it doesn't show up
in agent.log.
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >> The last message in agent.log
is a line saying we are
>>> right
>>> >>>> before
>>> >>>> >>> the
>>> >>>> >>> >>> >> call to this method.
>>> >>>> >>> >>> >>
>>> >>>> >>> >>> >>
>>> >>>> >>> >>>
>>> >>>> >>> >>
>>> >>>> >>> >>
>>> >>>> >>> >>
>>> >>>> >>> >> --
>>> >>>> >>> >> *Mike Tutkowski*
>>> >>>> >>> >> *Senior CloudStack Developer, SolidFire
Inc.*
>>> >>>> >>> >> e: mike.tutkowski@solidfire.com
>>> >>>> >>> >> o: 303.746.7302
>>> >>>> >>> >> Advancing the way the world uses the cloud<
>>> >>>> >>> http://solidfire.com/solution/overview/?video=play>
>>> >>>> >>> >> *™*
>>> >>>> >>> >>
>>> >>>> >>> >
>>> >>>> >>> >
>>> >>>> >>> >
>>> >>>> >>> > --
>>> >>>> >>> > *Mike Tutkowski*
>>> >>>> >>> > *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>> >>> > e: mike.tutkowski@solidfire.com
>>> >>>> >>> > o: 303.746.7302
>>> >>>> >>> > Advancing the way the world uses the
>>> >>>> >>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>> >>> > *™*
>>> >>>> >>>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> --
>>> >>>> >> *Mike Tutkowski*
>>> >>>> >> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>> >> e: mike.tutkowski@solidfire.com
>>> >>>> >> o: 303.746.7302
>>> >>>> >> Advancing the way the world uses the cloud<
>>> >>>> http://solidfire.com/solution/overview/?video=play>
>>> >>>> >> *™*
>>> >>>> >>
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > --
>>> >>>> > *Mike Tutkowski*
>>> >>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>> >>>> > e: mike.tutkowski@solidfire.com
>>> >>>> > o: 303.746.7302
>>> >>>> > Advancing the way the world uses the
>>> >>>> > cloud<http://solidfire.com/solution/overview/?video=play>
>>> >>>> > *™*
>>> >>>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> *Mike Tutkowski*
>>> >>> *Senior CloudStack Developer, SolidFire Inc.*
>>> >>> e: mike.tutkowski@solidfire.com
>>> >>> o: 303.746.7302
>>> >>> Advancing the way the world uses the cloud<
>>> http://solidfire.com/solution/overview/?video=play>
>>> >>> *™*
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> *Mike Tutkowski*
>>> >> *Senior CloudStack Developer, SolidFire Inc.*
>>> >> e: mike.tutkowski@solidfire.com
>>> >> o: 303.746.7302
>>> >> Advancing the way the world uses the
>>> >> cloud<http://solidfire.com/solution/overview/?video=play>
>>> >> *™*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message