qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eagy, Taylor" <te...@BlackbirdTech.com>
Subject RE: Python Connections Hang from Subprocesses
Date Wed, 07 Aug 2013 19:13:28 GMT
Oops. Let me clean it up a bit.

def attach(self):

Attach to the remote endpoint.
if not self._connected:
    self._connected = True
self._ewait(lambda: self._transport_connected and not self._unlinked())

def _ewait(self, predicate, timeout=None):

    result = self._wait(lambda: self.error or predicate(), timeout) <-- It times out here.
    return result

From: Eagy, Taylor [teagy@BlackbirdTech.com]
Sent: Wednesday, August 07, 2013 2:50 PM
To: users@qpid.apache.org
Subject: RE: Python Connections Hang from Subprocesses

Bill thanks for your response. The subprocess definitely runs when I don't try to connect
to the broker. Running a HTTPServer in it works too. It just doesn't connect to the qpid broker.
I did a little digging and I believe this is where it is hanging:

(In Connection class)

 270    @synchronized

 271 -<http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints-pysrc.html#>
 def attach<http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints.Connection-class.html#attach>(self):

 272      """
 273      Attach to the remote endpoint.
 274      """
 275      if not self._connected:
 276        self._connected = True
 277        self._driver.start<http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints-pysrc.html#>()
 278        self._wakeup()
 279      self._ewait(lambda: self._transport_connected and not self._unlinked())

The parent Python process can connect successfully, but the self._transport_connected never
gets set to True for all of the new Connection objects created in the subprocesses that are
trying to connect to the same broker. Where does this get set to True?
Could this have something to do with the predicate because it doesn't return an error it just
times out?

 212 -<http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints-pysrc.html#>
 def _ewait<http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints.Connection-class.html#_ewait>(self,
predicate, timeout=None):

 213      result = self._wait(lambda: self.error or predicate(), timeout<http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints-pysrc.html#>)
 214      self.check_error<http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints-pysrc.html#>()
 215      return result

From: Bill Freeman [ke1g.nh@gmail.com]
Sent: Wednesday, August 07, 2013 2:00 PM
To: users
Subject: Re: Python Connections Hang from Subprocesses

Subprocesses (using the subprocess module, or even the older exec stuff, as
opposed to threads, or even forked clones) are relatively trouble free in
python (except maybe on Windows, whose process model has that Microsoft
difference).  I've certainly made multiple connections to a broker from one
python process, as well as using tools like spout and drain, which are both
written in python, while my main development project is running connected.

Maybe there are broker configuration items which can affect this.  If so, I
hope that someone knowledgeable will speak up.  But I doubt that this is
the problem.

Are you sure that your subprocess runs?  It might be trying to report an
error to you.  Of, if you have pipes configured for interaction with the
invoking processor, it might be waiting on one of those.

You could, temporarily, instead of your intended code, have the subprocess
invoke something like BasicHTTPServer, and see if you can interact with it
using your browser.  If that also fails, it leaves the broker connection
out as the source of your problems.

Possibly easier is to have it log its arrival at various points, so you can
be sure where it is getting stuck.  If you've already confirmed that it's
in the broker connect, forgive me, and wait for a better answer.


On Wed, Aug 7, 2013 at 12:55 PM, Eagy, Taylor <teagy@blackbirdtech.com>wrote:

> Hello,
> I'm having an issue connecting multiple Python subprocesses to the qpid
> C++ broker. I have a main Python process that imports qpid.messaging at the
> top and spawns a few multiprocessing.Process objects that try to create a
> Qpid connection to the broker. Before I spawn the processes though, the
> main process creates a connection to the Qpid broker just fine, but all of
> the subprocess code hangs when it trys to connect to the qpid broker. I
> know qpid is supposed to be multi-threaded, but what about multiprocess
> with Python? What could be causing the connections to hang from the
> subprocesses? Is there some singleton object that isn't getting released
> that's blocking the other subprocesses from connecting?
> Thanks,
> TJ

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message