activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Posta <christian.po...@gmail.com>
Subject Re: Failed to fill batch / Restart 5.8 mkahadb / Connections overview missing
Date Tue, 04 Jun 2013 17:15:25 GMT
Maybe the kahadb files for that destination became corrupted somehow? Not
sure... is there any chance you can reliable reproduce this? Have you tried
setting ignore corrupted journal setting on kahadb to skip corrupted
sections when starting up the broker?

http://activemq.apache.org/kahadb.html


On Mon, Jun 3, 2013 at 5:53 AM, werab <enrico.rabeneck@idealo.de> wrote:

> Hello,
> we are running an activeMQ Server version 5.8 with a multi KahaDB
> configuration.
>
> <persistenceAdapter>
>     <mKahaDB directory="/activemq/data/kahadb">
>        <filteredPersistenceAdapters>
>           <filteredKahaDB perDestination="true" >
>              <persistenceAdapter>
>                 <kahaDB journalMaxFileLength="1mb"
>                         ignoreMissingJournalfiles="true"
>                         checkForCorruptJournalFiles="true"
>                         checksumJournalFiles="true"
>                         enableJournalDiskSyncs="false"/>
>              </persistenceAdapter>
>           </filteredKahaDB>
>        </filteredPersistenceAdapters>
>     </mKahaDB>
> </persistenceAdapter>
>
> We have a lot of different client connectors like Java JMS, Perl Stomp, PHP
> Stomp, etc.
>
> 1. Issue:
>
> After a few days of running a problem occur with some topics in combination
> with durable subscribers. It says something about “failed fill batch”, but
> I
> don't know what it means. After the first occurrence it is not possible for
> any producer to write to the topic nor to read from the topic by any
> consumer. Error message below:
>
> 2013-06-03 08:39:03,704 | ERROR |
> TopicStorePrefetch(IdealoMQ@sx119
> ,IPC-JMSCopyBoy-ENSLAVED_ON-ShopPropertyChange)
> ID:sx119.ipx-36620-1369825170384-1:1:4140813:1 -
> org.apache.activemq.broker.region.cursors.TopicStorePrefetch@5b58963
> :ShopPropertyChange,batchResetNeeded=false,storeHasMessages=true,size=5650,cacheEnabled=false,maxBatchSize:200
> - Failed to fill batch |
> org.apache.activemq.broker.region.cursors.AbstractStoreCursor | ActiveMQ
> Transport: tcp:///172.16.65.105:35386@61613
> java.lang.RuntimeException: java.io.EOFException
>         at
>
> org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:277)
>         at
>
> org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:110)
>         at
>
> org.apache.activemq.broker.region.cursors.StoreDurableSubscriberCursor.reset(StoreDurableSubscriberCursor.java:287)
>         at
>
> org.apache.activemq.broker.region.PrefetchSubscription.dispatchPending(PrefetchSubscription.java:628)
>         at
>
> org.apache.activemq.broker.region.DurableTopicSubscription.dispatchPending(DurableTopicSubscription.java:265)
>         at
>
> org.apache.activemq.broker.region.PrefetchSubscription.add(PrefetchSubscription.java:162)
>         at
>
> org.apache.activemq.broker.region.DurableTopicSubscription.add(DurableTopicSubscription.java:259)
>         at
>
> org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:48)
>         at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:688)
>         at
> org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:499)
>         at org.apache.activemq.broker.region.Topic.send(Topic.java:435)
>         at
>
> org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:114)
>         at
>
> org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:52)
>         at
>
> org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:406)
>         at
> org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:392)
>         at
>
> org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:282)
>         at
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129)
>         at
>
> org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
>         at
>
> org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:317)
>         at
>
> org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
>         at
>
> org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:499)
>         at
> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:749)
>         at
>
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:329)
>         at
>
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:184)
>         at
>
> org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:45)
>         at
>
> org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:288)
>         at
>
> org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:84)
>         at
>
> org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:195)
>         at
>
> org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:321)
>
> 2. Issue:
>
> The only chance to resolve the first issue is to restart the broker.
> Unfortunately, it is impossible for me to restart without deleting the
> whole
> kahadb directory. The error message tells me that there is something wrong
> with a topic directory, but when I remove the corresponding directory and
> try again it comes with the next one. Maybe I am missing/misreading  some
> parameters in the config file. Important parts of the error message below:
>
> 2013-06-03 09:50:40,351 | ERROR | Failed to load: class path resource
> [activemq.xml], reason: Error creating bean with name
> 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in class path
> resource [activemq.xml]: Invocation of init method failed; nested exception
> is java.lang.RuntimeException: Failed to start per destination persistence
> adapter for destination: topic#3a#2f#2fStoredOfferInvalidate, options:[]
>  | org.apache.activemq.xbean.XBeanBrokerFactory | WrapperSimpleAppMain
> org.springframework.beans.factory.BeanCreationException: Error creating
> bean
> with name 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in class
> path resource [activemq.xml]: Invocation of init method failed; nested
> exception is java.lang.RuntimeException: Failed to start per destination
> persistence adapter for destination: topic#3a#2f#2fStoredOfferInvalidate,
> options:[]
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1455)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
> ...
> ...
> Caused by: java.lang.RuntimeException: Failed to start per destination
> persistence adapter for destination: topic#3a#2f#2fStoredOfferInvalidate,
> options:[]
>         at
>
> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.startAdapter(MultiKahaDBPersistenceAdapter.java:183)
>         at
>
> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.registerExistingAdapter(MultiKahaDBPersistenceAdapter.java:343)
>         at
>
> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.findAndRegisterExistingAdapters(MultiKahaDBPersistenceAdapter.java:336)
>         at
>
> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.start(MultiKahaDBPersistenceAdapter.java:318)
>         at
>
> org.apache.activemq.broker.BrokerService.doStartPersistenceAdapter(BrokerService.java:598)
>
> 3. Issue:
>
> After updating to version 5.8 from 5.7 I am missing all current connections
> at the “Connection overview”. Apparently there are active connections,
> because the tools are running just fine and I can see them with jconsole
> per
> JMX. Do I have to switch something “on”?
>
> I appreciate any help. Maybe someone got related issues?
>
> Best regards
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Failed-to-fill-batch-Restart-5-8-mkahadb-Connections-overview-missing-tp4667740.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message