cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Tutkowski <mike.tutkow...@solidfire.com>
Subject Re: Managed storage with KVM
Date Sun, 22 Sep 2013 05:38:52 GMT
That's an interesting comment, Marcus.

It was my intent that it should work with any CloudStack "managed" storage
that uses an iSCSI target. Even though I'm using CHAP, I wrote the code so
CHAP didn't have to be used.

As I'm doing my testing, I can try to think about whether it is generic
enough to keep those names or not.

My expectation is that it is generic enough.


On Sat, Sep 21, 2013 at 11:32 PM, Marcus Sorensen <shadowsor@gmail.com>wrote:

> I added a comment to your diff. In general I think it looks good,
> though I obviously can't vouch for whether or not it will work. One
> thing I do have reservations about is the adaptor/pool naming. If you
> think the code is generic enough that it will work for anyone who does
> an iscsi LUN-per-volume plugin, then it's OK, but if there's anything
> about it that's specific to YOUR iscsi target or how it likes to be
> treated then I'd say that they should be named something less generic
> than iScsiAdmStorage.
>
> On Sat, Sep 21, 2013 at 7:23 PM, Mike Tutkowski
> <mike.tutkowski@solidfire.com> wrote:
> > Great - thanks!
> >
> > Just to give you an overview of what my code does (for when you get a
> > chance to review it):
> >
> > SolidFireHostListener is registered in SolidfirePrimaryDataStoreProvider.
> > Its hostConnect method is invoked when a host connects with the CS MS. If
> > the host is running KVM, the listener sends a ModifyStoragePoolCommand to
> > the host. This logic was based off of DefaultHostListener.
> >
> > The handling of ModifyStoragePoolCommand is unchanged. It invokes
> > createStoragePool on the KVMStoragePoolManager. The KVMStoragePoolManager
> > asks for an adaptor and finds my new one: iScsiAdmStorageAdaptor (which
> was
> > registered in the constructor for KVMStoragePoolManager under the key of
> > StoragePoolType.Iscsi.toString()).
> >
> > iScsiAdmStorageAdaptor.createStoragePool just makes an instance of
> > iScsiAdmStoragePool, adds it to a map, and returns the pointer to the
> > iScsiAdmStoragePool object. The key of the map is the UUID of the storage
> > pool.
> >
> > When a volume is attached, createPhysicalDisk is invoked for managed
> > storage rather than getPhysicalDisk. createPhysicalDisk uses iscsiadm to
> > establish the iSCSI connection to the volume on the SAN and a
> > KVMPhysicalDisk is returned to be used in the attach logic that follows.
> >
> > When a volume is detached, getPhysicalDisk is invoked with the IQN of the
> > volume if the storage pool in question is managed storage. Otherwise, the
> > normal vol.getPath() is used. iScsiAdmStorageAdaptor.getPhysicalDisk just
> > returns a new instance of KVMPhysicalDisk to be used in the detach logic.
> >
> > Once the volume has been detached, iScsiAdmStoragePool.deletePhysicalDisk
> > is invoked if the storage pool is managed. deletePhysicalDisk removes the
> > iSCSI connection to the volume using iscsiadm.
> >
> >
> > On Sat, Sep 21, 2013 at 5:46 PM, Marcus Sorensen <shadowsor@gmail.com
> >wrote:
> >
> >> Its the log4j properties file in /etc/cloudstack/agent change all INFO
> to
> >> DEBUG.  I imagine the agent just isn't starting, you can tail the log
> when
> >> you try to start the service, or maybe it will spit something out into
> one
> >> of the other files in /var/log/cloudstack/agent
> >> On Sep 21, 2013 5:19 PM, "Mike Tutkowski" <mike.tutkowski@solidfire.com
> >
> >> wrote:
> >>
> >> > This is how I've been trying to query for the status of the service (I
> >> > assume it could be started this way, as well, by changing "status" to
> >> > "start" or "restart"?):
> >> >
> >> > mtutkowski@ubuntu:/etc/cloudstack/agent$ sudo /usr/sbin/service
> >> > cloudstack-agent status
> >> >
> >> > I get this back:
> >> >
> >> > Failed to execute: * could not access PID file for cloudstack-agent
> >> >
> >> > I've made a bunch of code changes recently, though, so I think I'm
> going
> >> to
> >> > rebuild and redeploy everything.
> >> >
> >> > The debug info sounds helpful. Where can I set enable.debug?
> >> >
> >> > Thanks, Marcus!
> >> >
> >> >
> >> > On Sat, Sep 21, 2013 at 4:11 PM, Marcus Sorensen <shadowsor@gmail.com
> >> > >wrote:
> >> >
> >> > > OK, will check it out in the next few days. As mentioned, you can
> set
> >> up
> >> > > your Ubuntu vm as the management server as well if all else fails.
>  If
> >> > you
> >> > > can get to the mgmt server on 8250 from the KVM host, then you need
> to
> >> > > enable.debug on the agent. It won't run without complaining loudly
> if
> >> it
> >> > > can't get to the mgmt server, and I didn't see that in your agent
> log,
> >> so
> >> > > perhaps its not running. I assume you know how to stop/start the
> agent
> >> on
> >> > > KVM via 'service cloud stacks agent'.
> >> > > On Sep 21, 2013 3:02 PM, "Mike Tutkowski" <
> >> mike.tutkowski@solidfire.com>
> >> > > wrote:
> >> > >
> >> > > > Hey Marcus,
> >> > > >
> >> > > > I haven't yet been able to test my new code, but I thought you
> would
> >> > be a
> >> > > > good person to ask to review it:
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/mike-tutkowski/incubator-cloudstack/commit/ea74b312a8a36801994500407fd54f0cdda55e37
> >> > > >
> >> > > > All it is supposed to do is attach and detach a data disk (that
> has
> >> > > > guaranteed IOPS) with KVM as the hypervisor. The data disk
> happens to
> >> > be
> >> > > > from SolidFire-backed storage - where we have a 1:1 mapping
> between a
> >> > > > CloudStack volume and a data disk.
> >> > > >
> >> > > > There is no support for hypervisor snapshots or stuff like that
> >> > (likely a
> >> > > > future release)...just attaching and detaching a data disk in
4.3.
> >> > > >
> >> > > > Thanks!
> >> > > >
> >> > > >
> >> > > > On Fri, Sep 20, 2013 at 11:05 PM, Mike Tutkowski <
> >> > > > mike.tutkowski@solidfire.com> wrote:
> >> > > >
> >> > > > > When I re-deployed the DEBs, I didn't remove cloudstack-agent
> >> first.
> >> > > > Would
> >> > > > > that be a problem? I just did a sudo apt-get install
> >> > cloudstack-agent.
> >> > > > >
> >> > > > >
> >> > > > > On Fri, Sep 20, 2013 at 11:03 PM, Mike Tutkowski <
> >> > > > > mike.tutkowski@solidfire.com> wrote:
> >> > > > >
> >> > > > >> I get the same error running the command manually:
> >> > > > >>
> >> > > > >> mtutkowski@ubuntu:/etc/cloudstack/agent$ sudo
> /usr/sbin/service
> >> > > > >> cloudstack-agent status
> >> > > > >>  * could not access PID file for cloudstack-agent
> >> > > > >>
> >> > > > >>
> >> > > > >> On Fri, Sep 20, 2013 at 11:00 PM, Mike Tutkowski <
> >> > > > >> mike.tutkowski@solidfire.com> wrote:
> >> > > > >>
> >> > > > >>> agent.log looks OK to me:
> >> > > > >>>
> >> > > > >>> 2013-09-20 19:35:39,010 INFO  [cloud.agent.AgentShell]
> >> (main:null)
> >> > > > Agent
> >> > > > >>> started
> >> > > > >>> 2013-09-20 19:35:39,011 INFO  [cloud.agent.AgentShell]
> >> (main:null)
> >> > > > >>> Implementation Version is 4.3.0-SNAPSHOT
> >> > > > >>> 2013-09-20 19:35:39,015 INFO  [cloud.agent.AgentShell]
> >> (main:null)
> >> > > > >>> agent.properties found at
> /etc/cloudstack/agent/agent.properties
> >> > > > >>> 2013-09-20 19:35:39,023 INFO  [cloud.agent.AgentShell]
> >> (main:null)
> >> > > > >>> Defaulting to using properties file for storage
> >> > > > >>> 2013-09-20 19:35:39,024 INFO  [cloud.agent.AgentShell]
> >> (main:null)
> >> > > > >>> Defaulting to the constant time backoff algorithm
> >> > > > >>> 2013-09-20 19:35:39,029 INFO  [cloud.utils.LogUtils]
> (main:null)
> >> > > log4j
> >> > > > >>> configuration found at /etc/cloudstack/agent/log4j-cloud.xml
> >> > > > >>> 2013-09-20 19:35:39,163 INFO  [cloud.agent.Agent]
(main:null)
> id
> >> > is 3
> >> > > > >>> 2013-09-20 19:35:39,197 INFO
> >> > > > >>>  [resource.virtualnetwork.VirtualRoutingResource]
(main:null)
> >> > > > >>> VirtualRoutingResource _scriptDir to use:
> >> scripts/network/domr/kvm
> >> > > > >>>
> >> > > > >>> However, I wasn't aware that setup.log was important.
This
> seems
> >> to
> >> > > be
> >> > > > a
> >> > > > >>> problem, but I'm not sure what it might indicate:
> >> > > > >>>
> >> > > > >>> DEBUG:root:execute:sudo /usr/sbin/service cloudstack-agent
> status
> >> > > > >>> DEBUG:root:Failed to execute: * could not access
PID file for
> >> > > > >>> cloudstack-agent
> >> > > > >>> DEBUG:root:execute:sudo /usr/sbin/service cloudstack-agent
> start
> >> > > > >>>
> >> > > > >>>
> >> > > > >>> On Fri, Sep 20, 2013 at 10:53 PM, Marcus Sorensen
<
> >> > > shadowsor@gmail.com
> >> > > > >wrote:
> >> > > > >>>
> >> > > > >>>> Sorry, I saw that in the log, I thought it was
the agent log
> for
> >> > > some
> >> > > > >>>> reason. Is the agent started? That might be
the place to
> look.
> >> > There
> >> > > > is
> >> > > > >>>> an
> >> > > > >>>> agent log for the agent and one for the setup
when it adds
> the
> >> > host,
> >> > > > >>>> both
> >> > > > >>>> in /var/log
> >> > > > >>>> On Sep 20, 2013 10:42 PM, "Mike Tutkowski" <
> >> > > > >>>> mike.tutkowski@solidfire.com>
> >> > > > >>>> wrote:
> >> > > > >>>>
> >> > > > >>>> > Is it saying that the MS is at the IP address
or the KVM
> host?
> >> > > > >>>> >
> >> > > > >>>> > The KVM host is at 192.168.233.10.
> >> > > > >>>> > The MS host is at 192.168.233.1.
> >> > > > >>>> >
> >> > > > >>>> > I see this for my host Global Settings
parameter:
> >> > > > >>>> > hostThe ip address of management server192.168.233.1
> >> > > > >>>> >
> >> > > > >>>> >
> >> > > > >>>> > /etc/cloudstack/agent/agent.properties
has a
> >> host=192.168.233.1
> >> > > > value.
> >> > > > >>>> >
> >> > > > >>>> >
> >> > > > >>>> > On Fri, Sep 20, 2013 at 10:21 PM, Marcus
Sorensen <
> >> > > > >>>> shadowsor@gmail.com
> >> > > > >>>> > >wrote:
> >> > > > >>>> >
> >> > > > >>>> > > The log says your mgmt server is 192.168.233.10?
But you
> >> tried
> >> > > to
> >> > > > >>>> telnet
> >> > > > >>>> > to
> >> > > > >>>> > > 192.168.233.1? It might be enough
to change that in
> >> > > > >>>> > > /etc/cloudstack/agent/agent.properties,
but you may want
> to
> >> > edit
> >> > > > the
> >> > > > >>>> > config
> >> > > > >>>> > > as well to tell it the real ms IP.
> >> > > > >>>> > > On Sep 20, 2013 10:12 PM, "Mike Tutkowski"
<
> >> > > > >>>> mike.tutkowski@solidfire.com
> >> > > > >>>> > >
> >> > > > >>>> > > wrote:
> >> > > > >>>> > >
> >> > > > >>>> > > > Here's what my /etc/network/interfaces
file looks
> like, if
> >> > > that
> >> > > > >>>> is of
> >> > > > >>>> > > > interest (the 192.168.233.0 network
is the NAT network
> >> > VMware
> >> > > > >>>> Fusion
> >> > > > >>>> > set
> >> > > > >>>> > > > up):
> >> > > > >>>> > > >
> >> > > > >>>> > > > auto lo
> >> > > > >>>> > > > iface lo inet loopback
> >> > > > >>>> > > >
> >> > > > >>>> > > > auto eth0
> >> > > > >>>> > > > iface eth0 inet manual
> >> > > > >>>> > > >
> >> > > > >>>> > > > auto cloudbr0
> >> > > > >>>> > > > iface cloudbr0 inet static
> >> > > > >>>> > > >     address 192.168.233.10
> >> > > > >>>> > > >     netmask 255.255.255.0
> >> > > > >>>> > > >     network 192.168.233.0
> >> > > > >>>> > > >     broadcast 192.168.233.255
> >> > > > >>>> > > >     dns-nameservers 8.8.8.8
> >> > > > >>>> > > >     bridge_ports eth0
> >> > > > >>>> > > >     bridge_fd 5
> >> > > > >>>> > > >     bridge_stp off
> >> > > > >>>> > > >     bridge_maxwait 1
> >> > > > >>>> > > >     post-up route add default
gw 192.168.233.2 metric 1
> >> > > > >>>> > > >     pre-down route del default
gw 192.168.233.2
> >> > > > >>>> > > >
> >> > > > >>>> > > >
> >> > > > >>>> > > > On Fri, Sep 20, 2013 at 10:08
PM, Mike Tutkowski <
> >> > > > >>>> > > > mike.tutkowski@solidfire.com>
wrote:
> >> > > > >>>> > > >
> >> > > > >>>> > > > > You appear to be correct.
This is from the MS log
> >> (below).
> >> > > > >>>> Discovery
> >> > > > >>>> > > > timed
> >> > > > >>>> > > > > out.
> >> > > > >>>> > > > >
> >> > > > >>>> > > > > I'm not sure why this would
be. My network settings
> >> > > shouldn't
> >> > > > >>>> have
> >> > > > >>>> > > > changed
> >> > > > >>>> > > > > since the last time I tried
this.
> >> > > > >>>> > > > >
> >> > > > >>>> > > > > I am able to ping the KVM
host from the MS host and
> vice
> >> > > > versa.
> >> > > > >>>> > > > >
> >> > > > >>>> > > > > I'm even able to manually
kick off a VM on the KVM
> host
> >> > and
> >> > > > >>>> ping from
> >> > > > >>>> > > it
> >> > > > >>>> > > > > to the MS host.
> >> > > > >>>> > > > >
> >> > > > >>>> > > > > I am using NAT from my Mac
OS X host (also running
> the
> >> CS
> >> > > MS)
> >> > > > >>>> to the
> >> > > > >>>> > VM
> >> > > > >>>> > > > > running KVM (VMware Fusion).
> >> > > > >>>> > > > >
> >> > > > >>>> > > > > 2013-09-20 19:40:40,141
DEBUG
> >> > > > >>>> [c.c.h.k.d.LibvirtServerDiscoverer]
> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48)
Timeout,
> to
> >> > wait
> >> > > > for
> >> > > > >>>> the
> >> > > > >>>> > > host
> >> > > > >>>> > > > > connecting to mgt svr, assuming
it is failed
> >> > > > >>>> > > > > 2013-09-20 19:40:40,144
WARN
> >>  [c.c.r.ResourceManagerImpl]
> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48)
Unable to
> >> find
> >> > > the
> >> > > > >>>> server
> >> > > > >>>> > > > > resources at http://192.168.233.10
> >> > > > >>>> > > > > 2013-09-20 19:40:40,145
INFO
> >> >  [c.c.u.e.CSExceptionErrorCode]
> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48)
Could not
> >> find
> >> > > > >>>> exception:
> >> > > > >>>> > > > > com.cloud.exception.DiscoveryException
in error code
> >> list
> >> > > for
> >> > > > >>>> > > exceptions
> >> > > > >>>> > > > > 2013-09-20 19:40:40,147
WARN
>  [o.a.c.a.c.a.h.AddHostCmd]
> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48)
Exception:
> >> > > > >>>> > > > > com.cloud.exception.DiscoveryException:
Unable to add
> >> the
> >> > > host
> >> > > > >>>> > > > > at
> >> > > > >>>> > > > >
> >> > > > >>>> > > >
> >> > > > >>>> > >
> >> > > > >>>> >
> >> > > > >>>>
> >> > > >
> >> > >
> >> >
> >>
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:778)
> >> > > > >>>> > > > >
> >> > > > >>>> > > > > I do seem to be able to
telnet in from my KVM host to
> >> the
> >> > MS
> >> > > > >>>> host's
> >> > > > >>>> > > 8250
> >> > > > >>>> > > > > port:
> >> > > > >>>> > > > >
> >> > > > >>>> > > > > mtutkowski@ubuntu:~$ telnet
192.168.233.1 8250
> >> > > > >>>> > > > > Trying 192.168.233.1...
> >> > > > >>>> > > > > Connected to 192.168.233.1.
> >> > > > >>>> > > > > Escape character is '^]'.
> >> > > > >>>> > > > >
> >> > > > >>>> > > >
> >> > > > >>>> > > >
> >> > > > >>>> > > >
> >> > > > >>>> > > > --
> >> > > > >>>> > > > *Mike Tutkowski*
> >> > > > >>>> > > > *Senior CloudStack Developer,
SolidFire Inc.*
> >> > > > >>>> > > > e: mike.tutkowski@solidfire.com
> >> > > > >>>> > > > o: 303.746.7302
> >> > > > >>>> > > > Advancing the way the world uses
the
> >> > > > >>>> > > > cloud<
> http://solidfire.com/solution/overview/?video=play>
> >> > > > >>>> > > > *™*
> >> > > > >>>> > > >
> >> > > > >>>> > >
> >> > > > >>>> >
> >> > > > >>>> >
> >> > > > >>>> >
> >> > > > >>>> > --
> >> > > > >>>> > *Mike Tutkowski*
> >> > > > >>>> > *Senior CloudStack Developer, SolidFire
Inc.*
> >> > > > >>>> > e: mike.tutkowski@solidfire.com
> >> > > > >>>> > o: 303.746.7302
> >> > > > >>>> > Advancing the way the world uses the
> >> > > > >>>> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > > >>>> > *™*
> >> > > > >>>> >
> >> > > > >>>>
> >> > > > >>>
> >> > > > >>>
> >> > > > >>>
> >> > > > >>> --
> >> > > > >>> *Mike Tutkowski*
> >> > > > >>> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > > >>> e: mike.tutkowski@solidfire.com
> >> > > > >>> o: 303.746.7302
> >> > > > >>> Advancing the way the world uses the cloud<
> >> > > > http://solidfire.com/solution/overview/?video=play>
> >> > > > >>> *™*
> >> > > > >>>
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >> --
> >> > > > >> *Mike Tutkowski*
> >> > > > >> *Senior CloudStack Developer, SolidFire Inc.*
> >> > > > >> e: mike.tutkowski@solidfire.com
> >> > > > >> o: 303.746.7302
> >> > > > >> Advancing the way the world uses the cloud<
> >> > > > http://solidfire.com/solution/overview/?video=play>
> >> > > > >> *™*
> >> > > > >>
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > *Mike Tutkowski*
> >> > > > > *Senior CloudStack Developer, SolidFire Inc.*
> >> > > > > e: mike.tutkowski@solidfire.com
> >> > > > > o: 303.746.7302
> >> > > > > Advancing the way the world uses the cloud<
> >> > > > http://solidfire.com/solution/overview/?video=play>
> >> > > > > *™*
> >> > > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > *Mike Tutkowski*
> >> > > > *Senior CloudStack Developer, SolidFire Inc.*
> >> > > > e: mike.tutkowski@solidfire.com
> >> > > > o: 303.746.7302
> >> > > > Advancing the way the world uses the
> >> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > > > *™*
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > *Mike Tutkowski*
> >> > *Senior CloudStack Developer, SolidFire Inc.*
> >> > e: mike.tutkowski@solidfire.com
> >> > o: 303.746.7302
> >> > Advancing the way the world uses the
> >> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > *™*
> >> >
> >>
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message