uima-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jaroslaw Cwiklik <uim...@gmail.com>
Subject Re: Broken conections after ACTIVEMQ restart
Date Tue, 27 Sep 2016 16:42:06 GMT
Here is the uima-as 2.9.0-SNAPSHOT to try:
https://repository.apache.org/content/repositories/
snapshots/org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-
as-2.9.0-20160923.195233-6.tar.gz

Can you briefly describe your service? Does your service have any remote
delegates or all parts of the pipeline are in the same process?
Do you see and exceptions in the service log?

-jerry

On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera <nelsonrivera12@gmail.com>
wrote:

> I was testing with AMQ broker 5.14.0 and i get the same error. not
> appears to be broker
> issues, appears to be uima-as issues. Any other idea, i am stagnant at work
>
> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com>:
> > Can you install AMQ broker 5.14.0 to see if this is related to broker
> > issues.
> >
> > -jerry
> >
> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
> nelsonrivera12@gmail.com>
> > wrote:
> >
> >> just like that, the parent CAS comes back but the child CAS doesnt.
> >>
> >> And the child CAS if processed, because  the  uima-as log shows the
> >> lines of the other annotators primitive of the flow processing the
> >> child CAS, but not comes back to UIMA-AS client.
> >>
> >> uima-as log only shows
> >> "Jms Listener Failed. Endpoint:
> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed By:
> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> java.io.EOFException" ........
> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> >> Managed By: tcp://localhost:61616 Reason:
> >> javax.jms.IllegalStateException: The Consumer is closed..............
> >>
> >> The CAS is send to service after restart the Broker, and how say
> >> parent CAS comes back but the child CAS doesnt. And if i try quiesce
> >> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
> >> Remaining Number of CASes to Process:1".  I suppose that is the child
> >> CAS.
> >>
> >> the client if can send more CASes to the service, but not receive
> >> response, for each CAS send after the one CAS that was send after
> >> restart broker (for which  parent CAS comes back but the child CAS
> >> doesnt)  not comes back parent CAS or child CAS.
> >>
> >> More specific:
> >> scenario 1:
> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
> >>
> >> 2- Restart Broker (without restart the service).  UIMA-AS log show  :
> >> "Jms Listener Failed." ....
> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> >> comes back but the child CAS doesnt, to the uima-as client.)
> >> If i try quiesce the Aggregate Service console shows : "Quiescing
> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
> >>
> >> scenario 2:
> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
> >>
> >> 2- Restart Broker (without restart the service).  UIMA-AS log show  :
> >> "Jms Listener Failed." ....
> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> >> comes back but the child CAS doesnt, to the uima-as client)
> >> 4- Client  sends another CAS to UIMA-AS aggregate service.(not comes
> >> back parent CAS or child CAS , to the uima-as client)
> >> 5- Client  sends another CAS to UIMA-AS aggregate service.(not comes
> >> back parent CAS or child CAS , to the uima-as client)
> >>  If i try quiesce the Aggregate Service  console shows: "Quiescing
> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
> >>
> >>
> >>
> >>
> >>
> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com>:
> >> > When a parent CAS comes back (after broker is restored), can the
> client
> >> > send more CASes to the service and receive responses? I am trying to
> >> > determine what the real problem is here.
> >> >
> >> > 1) JMS Listener not recovering connection
> >> > 2) UIMA-AS is not handling CAS reply failures when broker connection
> is
> >> > stale
> >> >
> >> > -jerry
> >> >
> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <uimaee@gmail.com>
> >> > wrote:
> >> >
> >> >> Strange. You say that the parent CAS comes back but the child CAS
> >> doesnt.
> >> >> Are there any exceptions in the service log? Perhaps the service is
> >> >> unable
> >> >> to send the child CAS due to missing broker. There should be
> something
> >> in
> >> >> the uima-as log about this.
> >> >>
> >> >> -jerry
> >> >>
> >> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
> >> >> <nelsonrivera12@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Yes, it is supposed to do the service, given a
> >> >>> CAS it produces a child CAS and and this CAS continue in the flow,
> >> >>> that are other primitive annotators. And I expected that the child
> >> >>> CAS
> >> >>> will sent to  UIMA-AS client, but not do.
> >> >>>
> >> >>> the scenario:
> >> >>> 1- Start Broker and UIMA-AS aggregate service
> >> >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
> >> >>> everything is fine. The process is correct.
> >> >>>
> >> >>> 3- Restart Broker (without restart the service).  UIMA-AS log show
> :
> >> >>> "Jms Listener Failed." ....
> >> >>> 4- Client sends a CAS to UIMA-AS aggregate service
> >> >>>
> >> >>> UIMA-AS client receive the parent CAS, but not child CAS.
> >> >>>
> >> >>> Thank  for you quick reply
> >> >>>
> >> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com>:
> >> >>> > Can you clarify if the UIMA-AS service is a Cas Multiplier.
> Meaning
> >> >>> given a
> >> >>> > CAS it produces child CASes
> >> >>> > which are then sent to UIMA-AS client. Is this what this service
> is
> >> >>> > supposed to do if the broker is running?
> >> >>> >
> >> >>> > Can you confirm the scenario:
> >> >>> > 1) Client sends a CAS to UIMA-AS aggregate service
> >> >>> > 2) Service receives the CAS
> >> >>> > 3) Broker dies
> >> >>> > 4) Broker is restarted
> >> >>> >
> >> >>> > Does the UIMA-AS client receive the CAS back in a reply when
a
> >> >>> > broker
> >> >>> > is
> >> >>> > restarted?
> >> >>> >
> >> >>> > -Jerry
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
> >> >>> nelsonrivera12@gmail.com>
> >> >>> > wrote:
> >> >>> >
> >> >>> >> I will try the variants that you say. I was deepening
the
> problem:
> >> >>> >>
> >> >>> >> I have a Aggregate AE that contains CasMultiplier, the
original
> >> >>> >> CAS
> >> >>> >> is finish when it arrives to this CasMultiplier,  this
generates
> >> >>> >> new
> >> >>> >> CAS and is this CAS that continue in the flow of Aggregate.
> >> >>> >>
> >> >>> >> After restart broker (without restart the service of annotator)
> >> >>> >> and
> >> >>> >> send a CAS to process,  the listener associated to the
client api
> >> >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified
 when the
> >> >>> >> original CAS finalizes, but the new Cas generated in the
> >> >>> >> CasMultiplier, It is not notifies when finalizes.
> >> >>> >>
> >> >>> >> with Aggregate AE simple the log is the same, but how
the
> original
> >> >>> >> CAS
> >> >>> >>  is notifies into the listener when finalizes, not have
problem
> >> >>> >> because is not expected for another new CAS.
> >> >>> >>
> >> >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <uimaee@gmail.com>:
> >> >>> >> > If possible can you download AMQ 5.14.0 (latest)
and use that
> >> >>> >> > broker
> >> >>> >> > instead of the one the uima-as 2.8.1 ships with?
There was a
> >> >>> >> > nasty
> >> >>> bug
> >> >>> >> > in
> >> >>> >> > AMQ related to keep-alive protocol not working. I
am not sure
> if
> >> >>> >> > this
> >> >>> >> > is
> >> >>> >> > the cause but worth trying.
> >> >>> >> >
> >> >>> >> > You can download AMQ from here:
> >> >>> >> > http://activemq.apache.org/download.html
> >> >>> >> >
> >> >>> >> > -Jerry
> >> >>> >> >
> >> >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik
> >> >>> >> > <uimaee@gmail.com>
> >> >>> >> wrote:
> >> >>> >> >
> >> >>> >> >> Not sure what is causing this. I need to stage
this with
> >> >>> >> >> UIMA-AS
> >> >>> >> >> 2.8.1.
> >> >>> >> I
> >> >>> >> >> am currently testing with trunk (latest) code.
The most
> current
> >> >>> >> >> code
> >> >>> >> does
> >> >>> >> >> have newer broker.
> >> >>> >> >>
> >> >>> >> >> I've started a broker and a sample service:
> >> >>> >> >>
> >> >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh
> >> >>> >> >> $UIMA_HOME/examples/deploy/as/
> >> >>> >> >> Deploy_MeetingDetectorTAE.xml
> >> >>> >> >>
> >> >>> >> >> After the service started, I stopped the broker,
waited a bit,
> >> and
> >> >>> >> >> restarted the broker.
> >> >>> >> >>
> >> >>> >> >> With UIMA-AS from trunk I see the following in
the log:
> >> >>> >> >>
> >> >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
Controller:
> >> >>> >> >> Meeting
> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >>> >> >> Selector: Command=2000 OR Command=2002 Broker:
> >> >>> >> >> tcp://localhost:61616
> >> >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> SpringContainerDeployer.doStartListeners: INFO:
Controller:
> >> >>> >> >> Meeting
> >> >>> >> >> Detector TAE Trying to Start Listener on Endpoint:
> >> >>> >> >> queue://MeetingDetectorTaeQueue
> >> >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616
> >> >>> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
WARNING:
> >> Service:
> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
WARNING: Jms
> >> >>> Listener
> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed
By:
> >> >>> >> >> tcp://localhost:61616
> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >> >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> >> >>> >> WARNING:
> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
Unable To
> Connect
> >> To
> >> >>> >> >> Broker:
> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful
...
> >> >>> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
WARNING:
> >> Service:
> >> >>> >> >> Meeting Detector TAE Runtime Exception
> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.onException:
WARNING: Jms
> >> >>> Listener
> >> >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed
By:
> >> >>> >> >> tcp://localhost:61616
> >> >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException
> >> >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> >> >>> >> WARNING:
> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
Unable To
> Connect
> >> To
> >> >>> >> >> Broker:
> >> >>> >> >> tcp://localhost:61616 Retrying Until Successful
...
> >> >>> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> >> >>> >> WARNING:
> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
Recovered
> >> Connection
> >> >>> to
> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process
Again
> >> >>> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq.
> >> >>> >> >> UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> >> >>> >> WARNING:
> >> >>> >> >> Uima AS Service:Meeting Detector TAE Listener
Recovered
> >> Connection
> >> >>> to
> >> >>> >> >> Broker: tcp://localhost:61616 - Ready to Process
Again
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> I will repeat the above test with v 2.8.1
> >> >>> >> >>
> >> >>> >> >> -Jerry
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera
<
> >> >>> >> nelsonrivera12@gmail.com>
> >> >>> >> >> wrote:
> >> >>> >> >>
> >> >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what
happen?
> >> >>> >> >>>
> >> >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik
<
> uimaee@gmail.com
> >> >:
> >> >>> >> >>> > Which version of UIMA-AS are you using?
> >> >>> >> >>> >
> >> >>> >> >>> > -Jerry
> >> >>> >> >>> >
> >> >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson
rivera <
> >> >>> >> >>> nelsonrivera12@gmail.com>
> >> >>> >> >>> > wrote:
> >> >>> >> >>> >
> >> >>> >> >>> >> When start ActiveMQ and deploys
an annotator as service,
> >> >>> >> >>> >> processing
> >> >>> >> >>> >> is
> >> >>> >> >>> >> executed correctly, but  when the
broker is stopped and
> >> >>> >> >>> >> subsequently
> >> >>> >> >>> >> restarted. UIMA-AS log show the
lines below  and after
> this
> >> >>> >> >>> >> the
> >> >>> >> >>> >> api
> >> >>> >> >>> >> client uima-as is not notified in
the listener when a cas
> >> >>> >> >>> >> is
> >> >>> >> >>> >> processed, any more.
> >> >>> >> >>> >> can help me?:
> >> >>> >> >>> >>
> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate
Cluster Analyzer Runtime
> >> >>> Exception
> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
Endpoint:
> >> >>> >> XClusterAnalyzerAggregate
> >> >>> >> >>> >> Managed By: tcp://localhost:61616
Reason:
> >> >>> javax.jms.JMSException:
> >> >>> >> >>> >> java.io.EOFException
> >> >>> >> >>> >> 02:08:25.570 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> >> >>> nerContainer.
> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
Cluster Analyzer
> >> >>> >> >>> >> Listener
> >> >>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616
> Retrying
> >> >>> >> >>> >> ...
> >> >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener
Failed. Endpoint:
> >> >>> >> >>> >> temp-queue://ID:localhost-H81-
> M1-33610-1474481197190-1:1:1
> >> >>> Managed
> >> >>> >> >>> >> By:
> >> >>> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> >>> >> >>> >> java.io.EOFException
> >> >>> >> >>> >> 02:09:02.222 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> >> >>> nerContainer.
> >> >>> >> >>> >> handleListenerSetupFailure:
> >> >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate
Cluster Analyzer
> >> >>> >> >>> >> Listener
> >> >>> >> >>> >> Established Connection to Broker:
tcp://localhost:61616
> >> >>> >> >>> >>
> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate
Cluster Analyzer Runtime
> >> >>> Exception
> >> >>> >> >>> >> 02:09:02.317 - 14:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
Endpoint:
> >> >>> >> XClusterAnalyzerAggregate
> >> >>> >> >>> >> Managed By: tcp://localhost:61616
Reason:
> >> >>> >> >>> >> org.apache.activemq.ConnectionClosedException:
The
> >> connection
> >> >>> is
> >> >>> >> >>> >> already closed02:09:02.356 - 15:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Service: Aggregate
Cluster Analyzer Runtime
> >> >>> Exception
> >> >>> >> >>> >> 02:09:02.357 - 15:
> >> >>> >> >>> >> org.apache.uima.adapter.jms.activemq.
> UimaDefaultMessageListe
> >> >>> nerCont
> >> >>> >> >>> >> ainer.onException:
> >> >>> >> >>> >> ADVERTENCIA: Jms Listener Failed.
Endpoint:
> >> >>> >> XClusterAnalyzerAggregate
> >> >>> >> >>> >> Managed By: tcp://localhost:61616
Reason:
> >> >>> >> >>> >> javax.jms.IllegalStateException:
The Consumer is closed.
> >> >>> >> >>> >>
> >> >>> >> >>> >
> >> >>> >> >>>
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >
> >> >>> >>
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message