cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kelven Yang <kelven.y...@citrix.com>
Subject Re: [ACS41] Management server closes connection on port 8250
Date Wed, 03 Apr 2013 18:44:22 GMT


From: Sheng Yang <sheng@yasker.org<mailto:sheng@yasker.org>>
Date: Wednesday, April 3, 2013 10:44 AM
To: "<dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>" <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>,
Kelven Yang <kelven.yang@citrix.com<mailto:kelven.yang@citrix.com>>
Subject: Re: [ACS41] Management server closes connection on port 8250

On Tue, Apr 2, 2013 at 1:12 PM, Chip Childers <chip.childers@sungard.com<mailto:chip.childers@sungard.com>>
wrote:
On Tue, Apr 02, 2013 at 10:10:28PM +0200, Wido den Hollander wrote:
> Hi,
>
> On 04/02/2013 08:50 PM, Sheng Yang wrote:
> >On Tue, Apr 2, 2013 at 11:49 AM, Sheng Yang <sheng@yasker.org<mailto:sheng@yasker.org>>
wrote:
> >>On Tue, Apr 2, 2013 at 11:32 AM, Wido den Hollander <wido@widodh.nl<mailto:wido@widodh.nl>>
wrote:
> >>>Hi,
> >>>
> >>>Since I've upgraded my management server to 4.1 my agents refuse to connect,
> >>>their log says:
> >>>
> >>>2013-04-02 20:26:11,207 INFO  [utils.nio.NioClient] (Agent-Selector:null)
> >>>Connecting to 31.25.X.X:8250
> >>>2013-04-02 20:26:11,209 ERROR [utils.nio.NioConnection]
> >>>(Agent-Selector:null) Unable to initialize the threads.
> >>>java.io.IOException: SSL: Fail to init SSL! java.io.IOException: Connection
> >>>closed with -1 on reading size.
> >>>         at com.cloud.utils.nio.NioClient.init(NioClient.java:83)
> >>>         at com.cloud.utils.nio.NioConnection.run(NioConnection.java:108)
> >>>         at java.lang.Thread.run(Thread.java:679)
> >>>
> >>>So I tried a telnet connection:
> >>>
> >>>Connection closed by foreign host.
> >>>root@stack01:~# telnet 31.25.X.X 8250
> >>>Trying 31.25.X.X...
> >>>Connected to 31.X.X.X.
> >>>Escape character is '^]'.
> >>>Connection closed by foreign host.
> >>>root@stack01:~#
> >>>
> >>>So I didn't close the connection, but it was the management server.
> >>>
> >>>I cranked up the logging to DEBUG, but nothing shows in any of the logs,
so
> >>>I have no clue why this isn't working.
> >>>
> >>>On the mgmt server I see Java in LISTEN state on port 8250
> >>>
> >>>There is no firewall on the management server (it's my lab!).
> >>>
> >>>Any clues to what this could be before I start filing a in Jira? Since I'm
> >>>not sure if this is a bug.
> >>
> >>Have you upgrade your agent?
> >
>
> Yes, the Agent is also 4.1
>
> >Also you could try to enable TRACE for com.cloud.utils.nio to see more
> >log of NIO.
> >
>
> I think you mean at the agent?
>
> But what I think is weird is that the mgmt server directly closes
> the telnet connection as well and nothing shows up in the logs.
>
> In the mgmt server log I found this:
>
> 2013-04-02 22:08:41,156 INFO  [utils.nio.NioServer]
> (AgentManager-Selector:null) NioConnection started and listening on
> 0.0.0.0/0.0.0.0:8250<http://0.0.0.0/0.0.0.0:8250>
>
> What I did notice is that when I shutdown the mgmt server the Agent
> connects for a second:
>
> 2013-04-02 22:07:53,689 INFO  [utils.nio.NioClient]
> (Agent-Selector:null) Connecting to 31.25.X.X:8250
> 2013-04-02 22:07:53,934 INFO  [utils.nio.NioClient]
> (Agent-Selector:null) SSL: Handshake done
>
> And then the mgmt server exists and the connection gets lost.
>
> ANY clues about this weird behavior?
>
> Wido
>
> >--Sheng
> >>
> >>--Sheng
> >>>
> >>>Wido
>

Wido - Can you please open a blocker bug to track this?  It may end up
being a non-issue or something...  but I want to know that we have it
outstanding when considering when to cut the RC.

It remind me of one weird bug. The corrupted ssl keystore(due to some encryption/decryption
of db which I don't understand well, I remember Kelven knows more about it) on mgmt server
make it impossible to connect to mgmt server.


Yes, I encountered an issue in upgrade case where password related fields seem to be encrypted/corrupted
after the migration process.  It it worth to dig it out for a better explanation

-kelven



Could you clear "ssl.keystore" entry in configuration table of db, as well as cloud.keystore
file in the configuration path and restart the mgmt server and try again?

--Sheng

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message