cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ugo Vasi <ugo.v...@procne.it.INVALID>
Subject Re: secure hosts communications
Date Thu, 31 Jan 2019 11:26:28 GMT
Update:
by rebooting the host system, the libvirt is restarted and the ACS-agent 
has been reconnected to management.

The host remains in "unsecure" mode....

If I set to false "ca.plugin.root.auth.strictness" can I migrate the VM?



Il 31/01/19 11:50, Ugo Vasi ha scritto:
> Hi Rohit,
> I tryed renew certificate but it failed!
> Now libvirt does not restart and agent is disconnected:
>
> agent log:
> 2019-01-31 11:17:07,530 INFO 
> [resource.wrapper.LibvirtPostCertificateRenewalCommandWrapper] 
> (Certificate Renewal Timer:null) (logid:fe1554cc) Restarting libvirt 
> after certificate provisioning/renewal
> 2019-01-31 11:17:07,567 INFO  [cloud.agent.Agent] 
> (AgentShutdownThread:null) (logid:) Stopping the agent: Reason = sig.kill
> 2019-01-31 11:17:07,568 WARN  [cloud.agent.Agent] (Certificate Renewal 
> Timer:null) (logid:fe1554cc) Failed to execute post certificate 
> renewal command:
> java.lang.IllegalStateException: Shutdown in progress
>         at 
> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
>         at java.lang.Runtime.removeShutdownHook(Runtime.java:239)
>         at 
> com.cloud.agent.Agent$PostCertificateRenewalTask.runInContext(Agent.java:1157)
>         at 
> org.apache.cloudstack.managed.context.ManagedContextTimerTask$1.runInContext(ManagedContextTimerTask.java:30)
>         at 
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>         at 
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at 
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>         at 
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>         at 
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>         at 
> org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(ManagedContextTimerTask.java:32)
>         at java.util.TimerThread.mainLoop(Timer.java:555)
>         at java.util.TimerThread.run(Timer.java:505)
> 2019-01-31 11:17:09,797 INFO  [cloud.agent.AgentShell] (main:null) 
> (logid:) Agent started
> 2019-01-31 11:17:09,800 INFO  [cloud.agent.AgentShell] (main:null) 
> (logid:) Implementation Version is 4.11.2.0
> 2019-01-31 11:17:09,802 INFO  [cloud.agent.AgentShell] (main:null) 
> (logid:) agent.properties found at /etc/cloudstack/agent/agent.properties
> 2019-01-31 11:17:09,815 INFO  [cloud.agent.AgentShell] (main:null) 
> (logid:) Defaulting to using properties file for storage
> 2019-01-31 11:17:09,816 INFO  [cloud.agent.AgentShell] (main:null) 
> (logid:) Defaulting to the constant time backoff algorithm
> 2019-01-31 11:17:09,828 INFO  [cloud.utils.LogUtils] (main:null) 
> (logid:) log4j configuration found at 
> /etc/cloudstack/agent/log4j-cloud.xml
> 2019-01-31 11:17:09,850 INFO  [cloud.agent.AgentShell] (main:null) 
> (logid:) Using default Java settings for IPv6 preference for agent 
> connection
> 2019-01-31 11:17:09,998 INFO  [cloud.agent.Agent] (main:null) (logid:) 
> id is 5
> 2019-01-31 11:17:10,030 INFO  [kvm.resource.LibvirtConnection] 
> (main:null) (logid:) No existing libvirtd connection found. Opening a 
> new one
> 2019-01-31 11:17:10,175 ERROR [cloud.agent.AgentShell] (main:null) 
> (logid:) Unable to start agent:
> com.cloud.utils.exception.CloudRuntimeException: Failed to connect 
> socket to '/var/run/libvirt/libvirt-sock': No such file or directory
>         at 
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.configure(LibvirtComputingResource.java:914)
>         at com.cloud.agent.Agent.<init>(Agent.java:190)
>         at com.cloud.agent.AgentShell.launchNewAgent(AgentShell.java:453)
>         at 
> com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:422)
>         at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:406)
>         at com.cloud.agent.AgentShell.start(AgentShell.java:512)
>         at com.cloud.agent.AgentShell.main(AgentShell.java:547)
> (logs repeat)
>
> syslog:
>
>
> Jan 31 11:17:07 cshp214 sh[5065]: INFO 
> [resource.wrapper.LibvirtPostCertificateRenewalCommandWrapper] 
> (Certificate Renewal Timer:) (logid:fe1554cc) Restarting libvirt after 
> certificate provisioning/renewal
> Jan 31 11:17:07 cshp214 systemd[1]: Stopping CloudStack Agent...
> Jan 31 11:17:07 cshp214 sh[5065]: INFO  [cloud.agent.Agent] 
> (AgentShutdownThread:) (logid:) Stopping the agent: Reason = sig.kill
> Jan 31 11:17:07 cshp214 sh[5065]: WARN  [cloud.agent.Agent] 
> (Certificate Renewal Timer:) (logid:fe1554cc) Failed to execute post 
> certificate renewal command:
> Jan 31 11:17:07 cshp214 sh[5065]: java.lang.IllegalStateException: 
> Shutdown in progress
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> java.lang.Runtime.removeShutdownHook(Runtime.java:239)
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> com.cloud.agent.Agent$PostCertificateRenewalTask.runInContext(Agent.java:1157)
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> org.apache.cloudstack.managed.context.ManagedContextTimerTask$1.runInContext(ManagedContextTimerTask.java:30)
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(ManagedContextTimerTask.java:32)
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> java.util.TimerThread.mainLoop(Timer.java:555)
> Jan 31 11:17:07 cshp214 sh[5065]: #011at 
> java.util.TimerThread.run(Timer.java:505)
> Jan 31 11:17:08 cshp214 libvirtd[4700]: End of file while reading 
> data: Input/output error
> Jan 31 11:17:08 cshp214 libvirtd[4700]: End of file while reading 
> data: Input/output error
> Jan 31 11:17:08 cshp214 systemd[1]: Stopped CloudStack Agent.
> Jan 31 11:17:08 cshp214 systemd[1]: Stopping Virtualization daemon...
> Jan 31 11:17:08 cshp214 systemd[1]: Stopped Virtualization daemon.
> Jan 31 11:17:08 cshp214 systemd[1]: Starting Virtualization daemon...
> Jan 31 11:17:08 cshp214 systemd[1]: Started Virtualization daemon.
> Jan 31 11:17:08 cshp214 systemd[1]: Started CloudStack Agent.
> Jan 31 11:17:09 cshp214 sh[25387]: log4j:WARN No appenders could be 
> found for logger (com.cloud.agent.AgentShell).
> Jan 31 11:17:09 cshp214 sh[25387]: log4j:WARN Please initialize the 
> log4j system properly.
> Jan 31 11:17:09 cshp214 sh[25387]: log4j:WARN See 
> http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
> Jan 31 11:17:09 cshp214 sh[25387]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) Agent started
> Jan 31 11:17:09 cshp214 sh[25387]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) Implementation Version is 4.11.2.0
> Jan 31 11:17:09 cshp214 sh[25387]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) agent.properties found at 
> /etc/cloudstack/agent/agent.properties
> Jan 31 11:17:09 cshp214 sh[25387]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) Defaulting to using properties file for storage
> Jan 31 11:17:09 cshp214 sh[25387]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) Defaulting to the constant time backoff algorithm
> Jan 31 11:17:09 cshp214 sh[25387]: INFO  [cloud.utils.LogUtils] 
> (main:) (logid:) log4j configuration found at 
> /etc/cloudstack/agent/log4j-cloud.xml
> Jan 31 11:17:09 cshp214 sh[25387]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) Using default Java settings for IPv6 preference for 
> agent connection
> Jan 31 11:17:09 cshp214 sh[25387]: INFO  [cloud.agent.Agent] (main:) 
> (logid:) id is 5
> Jan 31 11:17:10 cshp214 sh[25387]: INFO 
> [kvm.resource.LibvirtConnection] (main:) (logid:) No existing libvirtd 
> connection found. Opening a new one
> -- 
> Jan 31 11:17:16 cshp214 snmpd[2460]: error on subcontainer 'ia_addr' 
> insert (-1)
> Jan 31 11:17:16 cshp214 snmpd[2460]: message repeated 3 times: [ error 
> on subcontainer 'ia_addr' insert (-1)]
> Jan 31 11:17:20 cshp214 systemd[1]: cloudstack-agent.service: Service 
> hold-off time over, scheduling restart.
> Jan 31 11:17:20 cshp214 systemd[1]: Stopped CloudStack Agent.
> Jan 31 11:17:20 cshp214 systemd[1]: Started CloudStack Agent.
> Jan 31 11:17:20 cshp214 sh[25457]: log4j:WARN No appenders could be 
> found for logger (com.cloud.agent.AgentShell).
> Jan 31 11:17:20 cshp214 sh[25457]: log4j:WARN Please initialize the 
> log4j system properly.
> Jan 31 11:17:20 cshp214 sh[25457]: log4j:WARN See 
> http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
> Jan 31 11:17:21 cshp214 sh[25457]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) Agent started
> Jan 31 11:17:21 cshp214 sh[25457]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) Implementation Version is 4.11.2.0
> Jan 31 11:17:21 cshp214 sh[25457]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) agent.properties found at 
> /etc/cloudstack/agent/agent.properties
> Jan 31 11:17:21 cshp214 sh[25457]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) Defaulting to using properties file for storage
> Jan 31 11:17:21 cshp214 sh[25457]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) Defaulting to the constant time backoff algorithm
> Jan 31 11:17:21 cshp214 sh[25457]: INFO  [cloud.utils.LogUtils] 
> (main:) (logid:) log4j configuration found at 
> /etc/cloudstack/agent/log4j-cloud.xml
> Jan 31 11:17:21 cshp214 sh[25457]: INFO  [cloud.agent.AgentShell] 
> (main:) (logid:) Using default Java settings for IPv6 preference for 
> agent connection
> Jan 31 11:17:21 cshp214 sh[25457]: INFO  [cloud.agent.Agent] (main:) 
> (logid:) id is 5
> Jan 31 11:17:21 cshp214 sh[25457]: INFO 
> [kvm.resource.LibvirtConnection] (main:) (logid:) No existing libvirtd 
> connection found. Opening a new one
> Jan 31 11:17:21 cshp214 sh[25457]: libvirt: XML-RPC error : Failed to 
> connect socket to '/var/run/libvirt/libvirt-sock': No such file or 
> directory
> Jan 31 11:17:21 cshp214 sh[25457]: ERROR [cloud.agent.AgentShell] 
> (main:) (logid:) Unable to start agent:
> Jan 31 11:17:21 cshp214 sh[25457]: 
> com.cloud.utils.exception.CloudRuntimeException: Failed to connect 
> socket to '/var/run/libvirt/libvirt-sock': No such file or directory
> Jan 31 11:17:21 cshp214 sh[25457]: #011at 
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.configure(LibvirtComputingResource.java:914)
> Jan 31 11:17:21 cshp214 sh[25457]: #011at 
> com.cloud.agent.Agent.<init>(Agent.java:190)
> Jan 31 11:17:21 cshp214 sh[25457]: #011at 
> com.cloud.agent.AgentShell.launchNewAgent(AgentShell.java:453)
> Jan 31 11:17:21 cshp214 sh[25457]: #011at 
> com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:422)
> Jan 31 11:17:21 cshp214 sh[25457]: #011at 
> com.cloud.agent.AgentShell.launchAgent(AgentShell.java:406)
> Jan 31 11:17:21 cshp214 sh[25457]: #011at 
> com.cloud.agent.AgentShell.start(AgentShell.java:512)
> Jan 31 11:17:21 cshp214 sh[25457]: #011at 
> com.cloud.agent.AgentShell.main(AgentShell.java:547)
> Jan 31 11:17:21 cshp214 sh[25457]: Unable to start agent: Failed to 
> connect socket to '/var/run/libvirt/libvirt-sock': No such file or 
> directory
> Jan 31 11:17:21 cshp214 systemd[1]: cloudstack-agent.service: Main 
> process exited, code=exited, status=67/n/a
> Jan 31 11:17:21 cshp214 systemd[1]: cloudstack-agent.service: Unit 
> entered failed state.
> Jan 31 11:17:21 cshp214 systemd[1]: cloudstack-agent.service: Failed 
> with result 'exit-code'.
> Jan 31 11:17:21 cshp214 dnsmasq[4000]: read /etc/hosts - 13 addresses
> Jan 31 11:17:21 cshp214 dnsmasq[4000]: read 
> /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> Jan 31 11:17:21 cshp214 dnsmasq-dhcp[4000]: read 
> /var/lib/libvirt/dnsmasq/default.hostsfile
> Jan 31 11:17:22 cshp214 snmpd[2460]: Connection from UDP: 
> [127.0.0.1]:37699->[127.0.0.1]:161
> Jan 31 11:17:24 cshp214 snmpd[2460]: message repeated 2 times: [ 
> Connection from UDP: [127.0.0.1]:37699->[127.0.0.1]:161]
> Jan 31 11:17:24 cshp214 libvirtd[25368]: libvirt version: 1.3.1, 
> package: 1ubuntu10.24 (Marc Deslauriers <marc.deslauriers@ubuntu.com> 
> Wed, 23 May 2018 13:29:29 -0400)
> Jan 31 11:17:24 cshp214 libvirtd[25368]: hostname: cshp214
> Jan 31 11:17:24 cshp214 libvirtd[25368]: Configured security driver 
> "none" disables default policy to create confined guests
> Jan 31 11:17:25 cshp214 libvirtd[25368]: unsupported configuration: 
> Security driver apparmor not enabled
>
>
> Can anyone help me?
>
> Il 30/01/19 13:37, Rohit Yadav ha scritto:
>>
>> Hi Ugo,
>>
>>
>> This will be a one-time procedure, and the KVM host and the VMs do 
>> not need a reboot but the provisionCertificate API will restart the 
>> libvirtd process (just check if that can have any side effects for 
>> your VMs/distro, on most modern distros restarting libvirtd does not 
>> have any side-effects on existing running VMs).
>>
>>
>> - Rohit
>>
>>
>>
>> rohit.yadav@shapeblue.com
>> www.shapeblue.com
>> @shapeblue
>>
>> ------------------------------------------------------------------------
>> *From:* Ugo Vasi <ugo.vasi@procne.it>
>> *Sent:* Wednesday, January 30, 2019 4:47:09 PM
>> *To:* users@cloudstack.apache.org; Rohit Yadav
>> *Subject:* Re: secure hosts communications
>> Hi Rohit,
>> I have a 4.11.2.0 ACS infrastructure (Ubuntu 16.04 with KVM hypervisor)
>> I see that all the hosts are in unsecure state from the UI and so the
>> live migration don't works (we had trubles with mgmt server).
>>
>> I read in the documentation that launching the provisionCertificate API
>> (by pressing the appropriate button in the UI) the certificates will be
>> renewed/regenerated for already connected agents/hosts.
>>
>> I do not understand if provisioning should be done manually on each host
>> or if the procedure should be done only once.
>>
>> Do this procedure reboot the host or the instances that it contains?
>>
>>
>> Thanks
>>
>>
>>
>> Il 27/11/18 09:49, Rohit Yadav ha scritto:
>> > Hi Richard,
>> >
>> >
>> > Please read: 
>> http://docs.cloudstack.apache.org/en/4.11.2.0/adminguide/hosts.html#security
>> >
>> >
>> > 4.11.2 is out, please consider using it instead of 4.11.1 as it has 
>> several bugfixes etc.
>> >
>> > In short, with all of your KVM hosts up and connected to mgmt 
>> server, first change the auth strictness global setting to true, then 
>> using API secure the hosts using the provisionCertificate API. In the 
>> UI, go to your hosts that don't show up as secure and click on the 
>> key button (a new button) to secure the host which calls the 
>> provisionCertificate API as well.
>> >
>> >
>> > - Rohit
>> >
>> > <https://cloudstack.apache.org>
>> >
>> >
>> >
>> > ________________________________
>> > From: Richard Persaud <richard.persaud@macys.com>
>> > Sent: Monday, November 26, 2018 8:19:56 PM
>> > To: users@cloudstack.apache.org
>> > Subject: RE: secure hosts communications
>> >
>> > Thank you, Rohit.
>> >
>> > I am using 4.11.1 with a full KVM environment. They are showing 
>> unsecure with strictness set to true.
>> >
>> > What configuration needs to be adjusted to have the KVM hosts show 
>> secure?
>> >
>> > Regards,
>> >
>> > Richard Persaud
>> >
>> > From: Rohit Yadav <rohit.yadav@shapeblue.com>
>> > Sent: Saturday, November 24, 2018 2:02 PM
>> > To: users@cloudstack.apache.org
>> > Subject: Re: secure hosts communications
>> >
>> > ⚠ EXT MSG:
>> >
>> > Richard,
>> >
>> >
>> > Starting 4.11, agent and management servers will use an in-built CA 
>> framework to secured hosts. Only in case of KVM hosts you may see an 
>> insecure state, otherwise all KVM hosts (agents) and SSVM/CPVM agents 
>> will by default in Up state will be secured. There is an auth 
>> strictness setting that should be true.
>> >
>> >
>> >
>> > - Rohit
>> >
>> > <https://cloudstack.apache.org>
>> >
>> >
>> >
>> > ________________________________
>> > From: Richard Persaud 
>> <richard.persaud@macys.com<mailto:richard.persaud@macys.com>>
>> > Sent: Saturday, November 24, 2018 4:21:24 AM
>> > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>
>> > Subject: secure hosts communications
>> >
>> > Hello,
>> >
>> > Is there straight-forward to enable secure communications between 
>> the management and the hosts?
>> >
>> > I have looked at many documentations but am still unable to get the 
>> hosts to show a "secure" state.
>> >
>> > Regards,
>> >
>> > Richard Persaud
>> >
>> >
>> > rohit.yadav@shapeblue.com<mailto:rohit.yadav@shapeblue.com>
>> > 
>> www.shapeblue.com<https://isolate.menlosecurity.com/0/eJyrViotylGyUsooKSmw0tcvLy_XK85ILEhNyilN1UvOz1XSUSrKV7Iy1FEqyUwBqjM0MFaqBQDf4BCe>
>> > Amadeus House, Floral Street, London  WC2E 9DPUK
>> > @shapeblue
>> >
>> >
>> >
>> >
>> > * This is an EXTERNAL EMAIL. Stop and think before clicking a link 
>> or opening attachments.
>> >
>> > rohit.yadav@shapeblue.com
>> > www.shapeblue.com <http://www.shapeblue.com>
>> > Amadeus House, Floral Street, London  WC2E 9DPUK
>> > @shapeblue
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>> -- 
>>
>> *Ugo Vasi* / System Administrator
>> ugo.vasi@procne.it <mailto:ugo.vasi@procne.it>
>>
>>
>>
>>
>> *Procne S.r.l.*
>> +39 0432 486 523
>> via Cotonificio, 45
>> 33010 Tavagnacco (UD)
>> www.procne.it <http://www.procne.it> <http://www.procne.it/>
>>
>>
>> Le informazioni contenute nella presente comunicazione ed i relativi
>> allegati possono essere riservate e sono, comunque, destinate
>> esclusivamente alle persone od alla Società sopraindicati. La
>> diffusione, distribuzione e/o copiatura del documento trasmesso da parte
>> di qualsiasi soggetto diverso dal destinatario è proibita sia ai sensi
>> dell'art. 616 c.p., che ai sensi del Decreto Legislativo n. 196/2003
>> "Codice in materia di protezione dei dati personali". Se avete ricevuto
>> questo messaggio per errore, vi preghiamo di distruggerlo e di informare
>> immediatamente Procne S.r.l. scrivendo all' indirizzo e-mail
>> info@procne.it <mailto:info@procne.it>.
>>
>
>


-- 

*Ugo Vasi* / System Administrator
ugo.vasi@procne.it <mailto:ugo.vasi@procne.it>




*Procne S.r.l.*
+39 0432 486 523
via Cotonificio, 45
33010 Tavagnacco (UD)
www.procne.it <http://www.procne.it/>


Le informazioni contenute nella presente comunicazione ed i relativi 
allegati possono essere riservate e sono, comunque, destinate 
esclusivamente alle persone od alla Società sopraindicati. La 
diffusione, distribuzione e/o copiatura del documento trasmesso da parte 
di qualsiasi soggetto diverso dal destinatario è proibita sia ai sensi 
dell'art. 616 c.p., che ai sensi del Decreto Legislativo n. 196/2003 
"Codice in materia di protezione dei dati personali". Se avete ricevuto 
questo messaggio per errore, vi preghiamo di distruggerlo e di informare 
immediatamente Procne S.r.l. scrivendo all' indirizzo e-mail 
info@procne.it <mailto:info@procne.it>.


Mime
View raw message