cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Girish Shilamkar <gir...@clogeny.com>
Subject Re: Http connection failure in marvin
Date Wed, 16 Oct 2013 05:53:18 GMT
I have been hitting this issue quite often since couple of weeks. 
I just switched the management server and so far haven't seen it.
Maybe updating httplib might help.

Regards,
Girish

On 16-Oct-2013, at 11:04 AM, Prasanna Santhanam <tsp@apache.org> wrote:

> On Wed, Oct 16, 2013 at 05:11:46AM +0000, Girish Shilamkar wrote:
>> Hello,
>> 
>> Sometimes while running automation testcases tests just fail with following error:
>> 
>> ConnectionError: HTTPConnectionPool(host='10.147.59.65', port=8080):
>> Max retries exceeded with url: /client/api?
>> apiKey=lcgWQ9XrjyIyPjsl7Y0BGI2udSRMs6S1w_42lWkwpCSfLWjkgoWvCakWwUCJ3TqfgMmuHWT4DXv5Lgoz-KCuSQ&egressdefaultpolicy=true&serviceproviderlist%5B2%5D.
>> provider=VirtualRouter&serviceproviderlist%5B5%5D.service=Dhcp&displaytext=Network+offering-VR+services-8FM4XI&specifyVlan=False&serviceproviderlist%5B4%5D.
>> provider=VirtualRouter&serviceproviderlist%5B1%5D.provider=VirtualRouter&availability=Optional&conservemode=True&servicecapabilitylist%5B1%5D.
>> capabilitytype=SupportedSourceNatTypes&serviceproviderlist%5B3%5D.service=Dns&serviceproviderlist%5B5%5D.provider=VirtualRouter&response=json&serviceproviderlist%5B0%5D.
>> provider=VirtualRouter&servicecapabilitylist%5B0%5D.capabilitytype=RedundantRouter&serviceproviderlist%5B8%5D.provider=VirtualRouter&serviceproviderlist%5B1%5D.
>> service=Lb&servicecapabilitylist%5B0%5D.service=SourceNat&serviceproviderlist%5B4%5D.
>> service=Firewall&supportedservices=Dhcp%2CDns%2CSourceNat%2CPortForwarding%2CVpn%2CFirewall%2CLb%2CUserData%2CStaticNat&traffictype=GUEST&servicecapabilitylist%5B1%5D.
>> service=SourceNat&serviceproviderlist%5B6%5D.provider=VirtualRouter&serviceproviderlist%5B8%5D.service=StaticNat&serviceproviderlist%5B3%5D.provider=VirtualRouter&name=Network+
>> offering-VR+services-OLND5Q&guestiptype=Isolated&serviceproviderlist%5B7%5D.provider=VirtualRouter&servicecapabilitylist%5B1%5D.
>> capabilityvalue=peraccount&serviceproviderlist%5B7%5D.service=SourceNat&serviceproviderlist%5B2%5D.service=PortForwarding&servicecapabilitylist%5B0%5D.
>> capabilityvalue=true&serviceproviderlist%5B6%5D.service=Vpn&command=createNetworkOffering&signature=dFBPTQoc%2B%2B42%2FhDVyQQOFB87c7k%3D&serviceproviderlist%5B0%5D.service=UserData
>> (Caused by <class 'httplib.BadStatusLine'>: '')
>> 
> 
> We actually don't use the submitCmdsAndWait to send the requests to
> the mgmt server. We just send it through the apiclient and
> cloudConnection.py takes over. I've seen the BadStatusLine issue with
> httplib on our test systems too. It is pretty random. At first I
> thought it was the throttling of the API but we don't enable that
> setting. I'm still looking for why this happens ... and how to repro
> it.
> 
> 
>> These errors are intermittent. While investigating this problem I found this in marvin.cloudstackTestClient
>> 
>> def submitCmdsAndWait(self, cmds, workers=1):
>>        '''FixME, httplib has issue if more than one thread submitted'''
>>        if self.asyncJobMgr is None:
>>            self.asyncJobMgr = asyncJobMgr.asyncJobMgr(self.apiClient,
>>                                                       self.dbConnection)
>>        return self.asyncJobMgr.submitCmdsAndWait(cmds, workers)
>> 
>> My theory is if the response from management server slow, multiple thread are submitted
and we hit this issue.
>> 
>> Please advise.
>> 
>> Regards,
>> Girish
>> 
> 
> -- 
> Prasanna.,
> 
> ------------------------
> Powered by BigRock.com
> 


Mime
View raw message