activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Burton <bur...@spinn3r.com>
Subject Re: leveldb issue
Date Thu, 18 Dec 2014 20:48:14 GMT
Hey.  I have a similar configuration and I’m getting a ton of "No reader
available for position” messages as well as significant data loss on AMQ
restart.

It literally loses about 95% of the messages I enqueued..

On Wed, Dec 17, 2014 at 9:06 AM, Christian Grassi <christiangrassi@gmail.com
> wrote:
>
> I have a 3 node 5.10 cluster with replicated levelDB as persistent store.
>
>
> <persistenceAdapter>
>          <replicatedLevelDB
>           directory="activemq-data"
>           replicas="3"
>           bind="tcp://0.0.0.0:0"
>           zkAddress="queue1:2181,queue2:2181,queue3:2181"
>           zkPath="/activemq/leveldb-stores"
>           hostname="queue3"
>           sync="quorum_disk"
>              />
>         </persistenceAdapter>
>
>
> I'm Stresstesting the queue with around 5000 persistent msg/s.
> 1 Producer
> 2 Consumers
>
> I get some warning messages on the log:
> On Master:
> 2014-12-16 16:25:28,375 | INFO  | Slave has disconnected:
> db6c9a23-7025-4384-b02f-dcda763113c3 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-3
> 2014-12-16 16:25:28,761 | WARN  | Unexpected session error:
> java.io.FileNotFoundException:
> /queue/activemq/conf/activemq-data/00000027b8d097b0.log (No such file or
> directory) | org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-1
> 2014-12-16 16:25:28,761 | INFO  | Slave has disconnected:
> 6a84579c-77fe-41eb-a728-1be472c12894 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-2
> 2014-12-16 16:25:29,603 | INFO  | Slave has connected:
> db6c9a23-7025-4384-b02f-dcda763113c3 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-2
> 2014-12-16 16:25:30,034 | INFO  | Slave has connected:
> 6a84579c-77fe-41eb-a728-1be472c12894 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-1
> 2014-12-16 16:25:31,360 | INFO  | Slave has now caught up:
> 6a84579c-77fe-41eb-a728-1be472c12894 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-4
> 2014-12-16 16:25:32,546 | INFO  | Slave has now caught up:
> db6c9a23-7025-4384-b02f-dcda763113c3 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-2
>
>
> On the slaves
>
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> I have a 3 node 5.10 cluster with replicated levelDB as persistent store.
>
>
> <persistenceAdapter>
>          <replicatedLevelDB
>           directory="activemq-data"
>           replicas="3"
>           bind="tcp://0.0.0.0:0"
>           zkAddress="queue1:2181,queue2:2181,queue3:2181"
>           zkPath="/activemq/leveldb-stores"
>           hostname="queue3"
>           sync="quorum_disk"
>              />
>         </persistenceAdapter>
>
>
> I'm Stresstesting the queue with around 5000 persistent msg/s.
> 1 Producer
> 2 Consumers
>
> I get some warning messages on the log:
> On Master:
> 2014-12-16 16:25:28,375 | INFO  | Slave has disconnected:
> db6c9a23-7025-4384-b02f-dcda763113c3 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-3
> 2014-12-16 16:25:28,761 | WARN  | Unexpected session error:
> java.io.FileNotFoundException:
> /queue/activemq/conf/activemq-data/00000027b8d097b0.log (No such file or
> directory) | org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-1
> 2014-12-16 16:25:28,761 | INFO  | Slave has disconnected:
> 6a84579c-77fe-41eb-a728-1be472c12894 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-2
> 2014-12-16 16:25:29,603 | INFO  | Slave has connected:
> db6c9a23-7025-4384-b02f-dcda763113c3 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-2
> 2014-12-16 16:25:30,034 | INFO  | Slave has connected:
> 6a84579c-77fe-41eb-a728-1be472c12894 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-1
> 2014-12-16 16:25:31,360 | INFO  | Slave has now caught up:
> 6a84579c-77fe-41eb-a728-1be472c12894 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-4
> 2014-12-16 16:25:32,546 | INFO  | Slave has now caught up:
> db6c9a23-7025-4384-b02f-dcda763113c3 |
> org.apache.activemq.leveldb.replicated.MasterLevelDBStore |
> hawtdispatch-DEFAULT-2
>
>
> On the slaves
>
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
> 2014-12-16 16:30:00,287 | WARN  | No reader available for position:
> 27fd52c562, log_infos:
>
> {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)}
> | org.apache.activemq.leveldb.RecordLog | Thread-16882
>


-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message