cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Yadav <rohit.ya...@shapeblue.com>
Subject [DISCUSS] How to fix failing VR-mgmt server links
Date Fri, 03 Apr 2015 09:52:51 GMT
Hi,

In large environments, one of the issues of a VM deployment or a network rule failing that
I find commonly is that the mgmt server is unable to reach to the VR via the host because
of network lag or issues between the host and the mgmt server. The sending operation on the
link is tried about 5 times (hardcoded) before it gives up and we see something like this
in the logs: "Unable to reach the peer that the agent is connected”.

Should we add a global setting to allow sysadmins to configure the agent link/socket (in various
AgentAttaches in engine/orchestration/src/com/cloud/agent/manager ?) timeout or please share
if something like this already exists or any other solution to this problem?

The other issue I see is that since VRs don’t have an agent running in it, to execute an
operation mgmt server SSH-es into it to run scripts, for a high load the number of open FDs
(so also TCP ports) on a VR/systemvm are limited which again can cause connections to fail/timeout
due to high number of requests VR is processing. A long term solution could be to implement
an agent (like ssvm/cpvm) that runs inside of the VR and talks to mgmt server over multiplexed
connection so we limit the number of connections from one mgmt server and we can get rid of
the SSH code and execution of barbaric scripts. Comments, suggestions, flames?

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +91 88 262 30892 | rohit.yadav@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>

This email and any attachments to it may be confidential and are intended solely for the use
of the individual to whom it is addressed. Any views or opinions expressed are solely those
of the author and do not necessarily represent those of Shape Blue Ltd or related companies.
If you are not the intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the sender if you believe
you have received this email in error. Shape Blue Ltd is a company incorporated in England
& Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated
in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company
registered by The Republic of South Africa and is traded under license from Shape Blue Ltd.
ShapeBlue is a registered trademark.
Mime
View raw message