qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eagy, Taylor" <te...@BlackbirdTech.com>
Subject RE: Python Connections Hang from Subprocesses
Date Wed, 07 Aug 2013 21:35:31 GMT
 What O/S are you running this on?



Ubuntu 12.04 LTS

What does the code that spawns the subprocess look like?



So my Process objects have a run() method that is called as the subprocess.



import qpid.messaging # at the top of the Python module



def run():

// some other non-qpid setup



conn = qpid.messaging.Connection(qpid_endpoint)

conn.open() <-- hangs here

self.sess = conn.session()


Can you, even when the subprocess is hung, run something like spout,
drain, or qpid-config, e.g.;

qpid-config queues

from another console window on the same box?



Yes, I can connect to the queue if I try to connect from another module (which runs a new
copy of the Python interpreter).

________________________________
From: Bill Freeman [ke1g.nh@gmail.com]
Sent: Wednesday, August 07, 2013 4:21 PM
To: users
Subject: Re: Python Connections Hang from Subprocesses

Just in case it makes me (or someone else) think of something else:

  What O/S are you running this on?

  What does the code that spawns the subprocess look like?

  Can you, even when the subprocess is hung, run something like spout,
drain, or qpid-config, e.g.;

      qpid-config queues

  from another console window on the same box?

Bill


On Wed, Aug 7, 2013 at 3:13 PM, Eagy, Taylor <teagy@blackbirdtech.com>wrote:

> Oops. Let me clean it up a bit.
>
> def attach(self):
>
> """
> Attach to the remote endpoint.
> """
> if not self._connected:
>     self._connected = True
>     self._driver.start()
>     self._wakeup()
> self._ewait(lambda: self._transport_connected and not self._unlinked())
>
> def _ewait(self, predicate, timeout=None):
>
>
>     result = self._wait(lambda: self.error or predicate(), timeout) <-- It
> times out here.
>     self.check_error()
>     return result
>
>
> ________________________________
> From: Eagy, Taylor [teagy@BlackbirdTech.com]
> Sent: Wednesday, August 07, 2013 2:50 PM
> To: users@qpid.apache.org
> Subject: RE: Python Connections Hang from Subprocesses
>
> Bill thanks for your response. The subprocess definitely runs when I don't
> try to connect to the broker. Running a HTTPServer in it works too. It just
> doesn't connect to the qpid broker. I did a little digging and I believe
> this is where it is hanging:
>
> (In Connection class)
>
>  270    @synchronized
>
>  271 -<
> http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints-pysrc.html#>
>  def attach<
> http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints.Connection-class.html#attach
> >(self):
>
>  272      """
>  273      Attach to the remote endpoint.
>  274      """
>  275      if not self._connected:
>  276        self._connected = True
>  277        self._driver.start<
> http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints-pysrc.html#
> >()
>  278        self._wakeup()
>  279      self._ewait(lambda: self._transport_connected and not
> self._unlinked())
>
> The parent Python process can connect successfully, but the
> self._transport_connected never gets set to True for all of the new
> Connection objects created in the subprocesses that are trying to connect
> to the same broker. Where does this get set to True?
> Could this have something to do with the predicate because it doesn't
> return an error it just times out?
>
>
>  212 -<
> http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints-pysrc.html#>
>  def _ewait<
> http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints.Connection-class.html#_ewait>(self,
> predicate, timeout=None):
>
>  213      result = self._wait(lambda: self.error or predicate(), timeout<
> http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints-pysrc.html#
> >)
>  214      self.check_error<
> http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/qpid.messaging.endpoints-pysrc.html#
> >()
>  215      return result
>
>
>
>
>
>
>
>
>
> Thanks,
> Taylor
> ________________________________
> From: Bill Freeman [ke1g.nh@gmail.com]
> Sent: Wednesday, August 07, 2013 2:00 PM
> To: users
> Subject: Re: Python Connections Hang from Subprocesses
>
> Subprocesses (using the subprocess module, or even the older exec stuff, as
> opposed to threads, or even forked clones) are relatively trouble free in
> python (except maybe on Windows, whose process model has that Microsoft
> difference).  I've certainly made multiple connections to a broker from one
> python process, as well as using tools like spout and drain, which are both
> written in python, while my main development project is running connected.
>
> Maybe there are broker configuration items which can affect this.  If so, I
> hope that someone knowledgeable will speak up.  But I doubt that this is
> the problem.
>
> Are you sure that your subprocess runs?  It might be trying to report an
> error to you.  Of, if you have pipes configured for interaction with the
> invoking processor, it might be waiting on one of those.
>
> You could, temporarily, instead of your intended code, have the subprocess
> invoke something like BasicHTTPServer, and see if you can interact with it
> using your browser.  If that also fails, it leaves the broker connection
> out as the source of your problems.
>
> Possibly easier is to have it log its arrival at various points, so you can
> be sure where it is getting stuck.  If you've already confirmed that it's
> in the broker connect, forgive me, and wait for a better answer.
>
> Bill
>
>
> On Wed, Aug 7, 2013 at 12:55 PM, Eagy, Taylor <teagy@blackbirdtech.com
> >wrote:
>
> > Hello,
> >
> >
> >
> > I'm having an issue connecting multiple Python subprocesses to the qpid
> > C++ broker. I have a main Python process that imports qpid.messaging at
> the
> > top and spawns a few multiprocessing.Process objects that try to create a
> > Qpid connection to the broker. Before I spawn the processes though, the
> > main process creates a connection to the Qpid broker just fine, but all
> of
> > the subprocess code hangs when it trys to connect to the qpid broker. I
> > know qpid is supposed to be multi-threaded, but what about multiprocess
> > with Python? What could be causing the connections to hang from the
> > subprocesses? Is there some singleton object that isn't getting released
> > that's blocking the other subprocesses from connecting?
> >
> >
> >
> > Thanks,
> >
> > TJ
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message