Mailing-List: contact cloudstack-users-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-users@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of Ahmad.Emneina@citrix.com
 designates 66.165.176.89 as permitted sender)
From: Ahmad Emneina <Ahmad.Emneina@citrix.com>
To: "cloudstack-users@incubator.apache.org"
	<cloudstack-users@incubator.apache.org>,
	"cloudstack-dev@incubator.apache.org" <cloudstack-dev@incubator.apache.org>
Date: Fri, 14 Sep 2012 10:09:37 -0700
Subject: Re: Xenserver 6.0.2/Cloudstack 3.0.2 stale socket files
Thread-Topic: Xenserver 6.0.2/Cloudstack 3.0.2 stale socket files
Thread-Index: Ac2Sm7byaSkBW4RnRKK41QjL0uNb4Q==
Message-ID: <CC78B2BE.10D13%Ahmad.Emneina@citrix.com>
In-Reply-To: <B479A0EB-36E8-4024-BA9D-97104E0C9FE2@me.com>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.0.0.100825
acceptlanguage: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

This looks like a prime candidate for a bug. There might be time to get it
in before 4.0 goes out!

On 9/14/12 9:54 AM, "Caleb Call" <calebcall@me.com> wrote:

>We came across an interesting issue yesterday in one of our clusters.  We
>ran out of inodes on all of our cluster members (since when does this
>happen in 2012?).  When this happened, it in turn made the / filesystem a
>read-only filesystem which in turn made all the hosts go in to emergency
>maintenance mode and as a result get marked down by Cloudstack.  We found
>that it was caused by hundreds of thousands of stale socket files in /tmp
>named "stream-unix.####.######".  To resolve the issue, we had to delete
>those stale socket files (find /tmp -name "*stream*" -mtime +7 -exec rm
>-v {} \;), then kill and restart xapi, then correct the emergency
>maintenance mode.  These hosts had only been up for 45 days before this
>issue occurred. =20
>
>In our scouring of the interwebs, the only other instance we've been able
>to find of this (or similar) happening is in the same setup we are
>currently running.  Xenserver 6.0.2 with CS 3.0.2.  Do these stream-unix
>sockets have anything to do with Cloudstack?  I would think if this was a
>Xenserver issue (bug), there would be a lot more on the internet about
>this happening.  For a temporary workaround, we've added a cronjob to
>cleanup these files but we'd really like to address the actual issue
>that's causing these sockets to become stale and not get cleaned-up.
>
>Thoughts?
>
>Thanks,
>Caleb
>


--=20
=C6