activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Tully (JIRA)" <j...@apache.org>
Subject [jira] Updated: (AMQ-1795) in use data files removed from data store under load
Date Thu, 12 Jun 2008 14:52:00 GMT

     [ https://issues.apache.org/activemq/browse/AMQ-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gary Tully updated AMQ-1795:
----------------------------

    Description: 
Under load, I see the error below.

the problem is that org.apache.activemq.store.amq.AMQPersistenceAdapter.cleanup() does not capture all of the in use files.
I have made some changes that improves the situation but there is still a window, that seems to be related to spooling messages when a memory limit is reached.


2008-06-12 14:55:16,689 [main           ] INFO  BrokerService                  - Using Persistence Adapter: AMQPersistenceAdapter(activemq-data\localhost)
2008-06-12 14:55:16,704 [main           ] INFO  AMQPersistenceAdapter          - AMQStore starting using directory: activemq-data\localhost
2008-06-12 14:55:17,361 [main           ] INFO  AMQPersistenceAdapter          - Journal deleted: 
2008-06-12 14:55:17,361 [main           ] INFO  KahaStore                      - Kaha Store successfully deleted data directory activemq-data\localhost\kr-store\data
2008-06-12 14:55:17,392 [main           ] INFO  KahaStore                      - Kaha Store successfully deleted data directory activemq-data\localhost\kr-store\state
2008-06-12 14:55:17,392 [main           ] INFO  KahaStore                      - Kaha Store using data directory activemq-data\localhost\kr-store\state
2008-06-12 14:55:17,470 [main           ] INFO  AMQPersistenceAdapter          - Active data files: []
2008-06-12 14:55:17,705 [main           ] INFO  BrokerService                  - ActiveMQ null JMS Message Broker (localhost) is starting
2008-06-12 14:55:17,705 [main           ] INFO  BrokerService                  - For help or more information please see: http://activemq.apache.org/
2008-06-12 14:55:17,814 [JMX connector  ] INFO  ManagementContext              - JMX consoles can connect to service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
2008-06-12 14:55:17,924 [main           ] INFO  KahaStore                      - Kaha Store using data directory activemq-data\localhost\kr-store\data
2008-06-12 14:55:17,939 [main           ] INFO  TransportServerThreadSupport   - Listening for connections at: tcp://salthill:61616
2008-06-12 14:55:17,939 [main           ] INFO  TransportConnector             - Connector Default Started
2008-06-12 14:55:17,939 [main           ] INFO  BrokerService                  - ActiveMQ JMS Message Broker (localhost, ID:salthill-2790-1213278917752-0:0) started
2008-06-12 14:55:17,939 [main           ] INFO  MissingDataFileTest            - Starting broker..
2008-06-12 14:55:58,516 [age Thread Pool] ERROR AsyncDataManager               - Looking for key 551 but not found in fileMap: {1014=data-1014 number = 1014 , length = 1960 refCount = 6, 607=data-607 number = 607 , length = 1266 refCount = 1, 928=data-928 number = 928 , .. .. 622 , length = 1352 refCount = 2, 328=data-328 number = 328 , length = 1266 refCount = 1, 256=data-256 number = 256 , length = 1264 refCount = 1, 92=data-92 number = 92 , length = 1264 refCount = 1, 404=data-404 number = 404 , length = 1266 refCount = 1}
2008-06-12 14:55:58,563 [age Thread Pool] ERROR AbstractStoreCursor            - Failed to fill batch
java.io.IOException: Failed to read to journal for: offset = 0, file = 551, size = -1, type = 0. Reason: java.io.IOException: Could not locate data file data--551
	at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:33)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.createReadException(AMQPersistenceAdapter.java:635)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:521)
	at org.apache.activemq.store.amq.AMQMessageStore.getMessage(AMQMessageStore.java:432)
	at org.apache.activemq.store.amq.RecoveryListenerAdapter.recoverMessageReference(RecoveryListenerAdapter.java:54)
	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverReference(KahaReferenceStore.java:82)
	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverNextMessages(KahaReferenceStore.java:120)
	at org.apache.activemq.store.amq.AMQMessageStore.recoverNextMessages(AMQMessageStore.java:530)
	at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:75)
	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:188)
	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.onUsageChanged(AbstractStoreCursor.java:157)
	at org.apache.activemq.usage.Usage$1.run(Usage.java:266)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
	at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.IOException: Could not locate data file data--551
	at org.apache.activemq.kaha.impl.async.AsyncDataManager.getDataFile(AsyncDataManager.java:303)
	at org.apache.activemq.kaha.impl.async.AsyncDataManager.read(AsyncDataManager.java:613)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:518)
	... 12 more
2008-06-12 14:55:58,563 [age Thread Pool] ERROR AbstractStoreCursor            - Failed to fill batch 
java.lang.RuntimeException: java.io.IOException: Failed to read to journal for: offset = 0, file = 551, size = -1, type = 0. Reason: java.io.IOException: Could not locate data file data--551
	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:191)
	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.onUsageChanged(AbstractStoreCursor.java:157)
	at org.apache.activemq.usage.Usage$1.run(Usage.java:266)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
	at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.IOException: Failed to read to journal for: offset = 0, file = 551, size = -1, type = 0. Reason: java.io.IOException: Could not locate data file data--551
	at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:33)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.createReadException(AMQPersistenceAdapter.java:635)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:521)
	at org.apache.activemq.store.amq.AMQMessageStore.getMessage(AMQMessageStore.java:432)
	at org.apache.activemq.store.amq.RecoveryListenerAdapter.recoverMessageReference(RecoveryListenerAdapter.java:54)
	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverReference(KahaReferenceStore.java:82)
	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverNextMessages(KahaReferenceStore.java:120)
	at org.apache.activemq.store.amq.AMQMessageStore.recoverNextMessages(AMQMessageStore.java:530)
	at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:75)
	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:188)
	... 5 more
Caused by: java.io.IOException: Could not locate data file data--551
	at org.apache.activemq.kaha.impl.async.AsyncDataManager.getDataFile(AsyncDataManager.java:303)
	at org.apache.activemq.kaha.impl.async.AsyncDataManager.read(AsyncDataManager.java:613)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:518)
	... 12 more
2008-06-12 15:00:18,531 [main           ] INFO  BrokerService                  - ActiveMQ Message Broker (localhost, ID:salthill-2790-1213278917752-0:0) is shutting down
2008-06-12 15:00:20,438 [main           ] INFO  TransportConnector             - Connector Default Stopped
2008-06-12 15:00:20,641 [main           ] INFO  BrokerService                  - ActiveMQ JMS Message Broker (localhost, ID:salthill-2790-1213278917752-0:0) stopped


  was:
Under load, I see the error below.

the problem is that org.apache.activemq.store.amq.AMQPersistenceAdapter.cleanup() does not capture all of the in use files.
I have made some changes that improves the situation but there is still a window, that seems to be related to spooling messages when a memory limit is reached.


2008-06-12 14:55:16,689 [main           ] INFO  BrokerService                  - Using Persistence Adapter: AMQPersistenceAdapter(activemq-data\localhost)
2008-06-12 14:55:16,704 [main           ] INFO  AMQPersistenceAdapter          - AMQStore starting using directory: activemq-data\localhost
2008-06-12 14:55:17,361 [main           ] INFO  AMQPersistenceAdapter          - Journal deleted: 
2008-06-12 14:55:17,361 [main           ] INFO  KahaStore                      - Kaha Store successfully deleted data directory activemq-data\localhost\kr-store\data
2008-06-12 14:55:17,392 [main           ] INFO  KahaStore                      - Kaha Store successfully deleted data directory activemq-data\localhost\kr-store\state
2008-06-12 14:55:17,392 [main           ] INFO  KahaStore                      - Kaha Store using data directory activemq-data\localhost\kr-store\state
2008-06-12 14:55:17,470 [main           ] INFO  AMQPersistenceAdapter          - Active data files: []
2008-06-12 14:55:17,705 [main           ] INFO  BrokerService                  - ActiveMQ null JMS Message Broker (localhost) is starting
2008-06-12 14:55:17,705 [main           ] INFO  BrokerService                  - For help or more information please see: http://activemq.apache.org/
2008-06-12 14:55:17,814 [JMX connector  ] INFO  ManagementContext              - JMX consoles can connect to service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
2008-06-12 14:55:17,924 [main           ] INFO  KahaStore                      - Kaha Store using data directory activemq-data\localhost\kr-store\data
2008-06-12 14:55:17,939 [main           ] INFO  TransportServerThreadSupport   - Listening for connections at: tcp://salthill:61616
2008-06-12 14:55:17,939 [main           ] INFO  TransportConnector             - Connector Default Started
2008-06-12 14:55:17,939 [main           ] INFO  BrokerService                  - ActiveMQ JMS Message Broker (localhost, ID:salthill-2790-1213278917752-0:0) started
2008-06-12 14:55:17,939 [main           ] INFO  MissingDataFileTest            - Starting broker..
2008-06-12 14:55:58,516 [age Thread Pool] ERROR AsyncDataManager               - Looking for key 551 but not found in fileMap: {1014=data-1014 number = 1014 , length = 1960 refCount = 6, 607=data-607 number = 607 , length = 1266 refCount = 1, 928=data-928 number = 928 , length = 1433 refCount = 3, 415=data-415 number = 415 , length = 1266 refCount = 1, 1049=data-1049 number = 1049 , length = 1766 refCount = 7, 672=data-672 number = 672 , length = 1312 refCount = 1, 159=data-159 number = 159 , length = 1264 refCount = 1, 86=data-86 number = 86 , length = 1264 refCount = 1, 285=data-285 number = 285 , length = 1352 refCount = 2, 224=data-224 number = 224 , length = 1433 refCount = 3, 798=data-798 number = 798 , length = 1264 refCount = 1, 905=data-905 number = 905 , length = 1264 refCount = 1, 29=data-29 number = 29 , length = 1266 refCount = 1, 542=data-542 number = 542 , length = 1264 refCount = 1, 460=data-460 number = 460 , length = 1345 refCount = 2, 481=data-481 number = 481 , length = 1264 refCount = 1, 501=data-501 number = 501 , length = 1264 refCount = 1, 603=data-603 number = 603 , length = 1264 refCount = 1, 1045=data-1045 number = 1045 , length = 2011 refCount = 7, 995=data-995 number = 995 , length = 1960 refCount = 6, 859=data-859 number = 859 , length = 1264 refCount = 1, 733=data-733 number = 733 , length = 1326 refCount = 2, 375=data-375 number = 375 , length = 1433 refCount = 3, 706=data-706 number = 706 , length = 1264 refCount = 1, 212=data-212 number = 212 , length = 1264 refCount = 1, 949=data-949 number = 949 , length = 1433 refCount = 3, 1041=data-1041 number = 1041 , length = 1880 refCount = 6, 232=data-232 number = 232 , length = 1264 refCount = 1, 94=data-94 number = 94 , length = 1352 refCount = 2, 771=data-771 number = 771 , length = 1324 refCount = 2, 310=data-310 number = 310 , length = 1264 refCount = 1, 599=data-599 number = 599 , length = 1412 refCount = 3, 350=data-350 number = 350 , length = 1345 refCount = 2, 888=data-888 number = 888 , length = 1266 refCount = 1, 718=data-718 number = 718 , length = 1264 refCount = 1, 371=data-371 number = 371 , length = 1264 refCount = 1, 547=data-547 number = 547 , length = 1264 refCount = 1, 33=data-33 number = 33 , length = 1264 refCount = 1, 1059=data-1059 number = 1059 , length = 2011 refCount = 7, 823=data-823 number = 823 , length = 1264 refCount = 1, 330=data-330 number = 330 , length = 1264 refCount = 1, 281=data-281 number = 281 , length = 1595 refCount = 2, 204=data-204 number = 204 , length = 1264 refCount = 1, 794=data-794 number = 794 , length = 1324 refCount = 2, 970=data-970 number = 970 , length = 1433 refCount = 3, 783=data-783 number = 783 , length = 1264 refCount = 1, 420=data-420 number = 420 , length = 1264 refCount = 1, 456=data-456 number = 456 , length = 1326 refCount = 2, 714=data-714 number = 714 , length = 1343 refCount = 2, 527=data-527 number = 527 , length = 1593 refCount = 2, 74=data-74 number = 74 , length = 1343 refCount = 2, 261=data-261 number = 261 , length = 1266 refCount = 1, 25=data-25 number = 25 , length = 1345 refCount = 2, 538=data-538 number = 538 , length = 1391 refCount = 2, 865=data-865 number = 865 , length = 1264 refCount = 1, 844=data-844 number = 844 , length = 1266 refCount = 1, 876=data-876 number = 876 , length = 1352 refCount = 2, 497=data-497 number = 497 , length = 1264 refCount = 1, 653=data-653 number = 653 , length = 1431 refCount = 3, 241=data-241 number = 241 , length = 1433 refCount = 3, 208=data-208 number = 208 , length = 1266 refCount = 1, 444=data-444 number = 444 , length = 1352 refCount = 2, 953=data-953 number = 953 , length = 1544 refCount = 5, 779=data-779 number = 779 , length = 1264 refCount = 1, 710=data-710 number = 710 , length = 1354 refCount = 2, 729=data-729 number = 729 , length = 1312 refCount = 1, 1067=data-1067 number = 1067 , length = 2011 refCount = 7, 966=data-966 number = 966 , length = 1326 refCount = 2, 555=data-555 number = 555 , length = 1264 refCount = 1, 592=data-592 number = 592 , length = 1266 refCount = 1, 273=data-273 number = 273 , length = 1264 refCount = 1, 786=data-786 number = 786 , length = 1412 refCount = 3, 359=data-359 number = 359 , length = 1264 refCount = 1, 754=data-754 number = 754 , length = 1264 refCount = 1, 668=data-668 number = 668 , length = 1431 refCount = 3, 616=data-616 number = 616 , length = 1431 refCount = 3, 530=data-530 number = 530 , length = 1264 refCount = 1, 869=data-869 number = 869 , length = 1266 refCount = 1, 815=data-815 number = 815 , length = 1264 refCount = 1, 102=data-102 number = 102 , length = 1264 refCount = 1, 934=data-934 number = 934 , length = 1264 refCount = 1, 677=data-677 number = 677 , length = 1264 refCount = 1, 269=data-269 number = 269 , length = 1314 refCount = 1, 277=data-277 number = 277 , length = 1264 refCount = 1, 696=data-696 number = 696 , length = 1264 refCount = 1, 790=data-790 number = 790 , length = 1264 refCount = 1, 612=data-612 number = 612 , length = 1266 refCount = 1, 1022=data-1022 number = 1022 , length = 1960 refCount = 6, 689=data-689 number = 689 , length = 1372 refCount = 2, 180=data-180 number = 180 , length = 1324 refCount = 2, 534=data-534 number = 534 , length = 1264 refCount = 1, 265=data-265 number = 265 , length = 1354 refCount = 2, 78=data-78 number = 78 , length = 1326 refCount = 2, 436=data-436 number = 436 , length = 1314 refCount = 1, 945=data-945 number = 945 , length = 1312 refCount = 1, 183=data-183 number = 183 , length = 1343 refCount = 2, 346=data-346 number = 346 , length = 1264 refCount = 1, 139=data-139 number = 139 , length = 1324 refCount = 2, 595=data-595 number = 595 , length = 1264 refCount = 1, 808=data-808 number = 808 , length = 1343 refCount = 2, 1018=data-1018 number = 1018 , length = 2020 refCount = 7, 851=data-851 number = 851 , length = 1264 refCount = 1, 880=data-880 number = 880 , length = 1312 refCount = 1, 649=data-649 number = 649 , length = 1324 refCount = 2, 509=data-509 number = 509 , length = 1352 refCount = 2, 366=data-366 number = 366 , length = 1312 refCount = 1, 574=data-574 number = 574 , length = 1264 refCount = 1, 440=data-440 number = 440 , length = 1343 refCount = 2, 941=data-941 number = 941 , length = 1433 refCount = 3, 960=data-960 number = 960 , length = 1414 refCount = 3, 635=data-635 number = 635 , length = 1431 refCount = 3, 171=data-171 number = 171 , length = 1653 refCount = 3, 891=data-891 number = 891 , length = 1266 refCount = 1, 571=data-571 number = 571 , length = 1264 refCount = 1, 342=data-342 number = 342 , length = 1264 refCount = 1, 123=data-123 number = 123 , length = 1264 refCount = 1, 318=data-318 number = 318 , length = 1391 refCount = 2, 704=data-704 number = 704 , length = 1266 refCount = 1, 118=data-118 number = 118 , length = 1345 refCount = 2, 682=data-682 number = 682 , length = 1264 refCount = 1, 937=data-937 number = 937 , length = 1264 refCount = 1, 913=data-913 number = 913 , length = 1264 refCount = 1, 747=data-747 number = 747 , length = 1264 refCount = 1, 379=data-379 number = 379 , length = 1264 refCount = 1, 314=data-314 number = 314 , length = 1412 refCount = 3, 338=data-338 number = 338 , length = 1264 refCount = 1, 739=data-739 number = 739 , length = 1264 refCount = 1, 245=data-245 number = 245 , length = 1312 refCount = 1, 757=data-757 number = 757 , length = 1264 refCount = 1, 693=data-693 number = 693 , length = 1264 refCount = 1, 407=data-407 number = 407 , length = 1264 refCount = 1, 428=data-428 number = 428 , length = 1264 refCount = 1, 514=data-514 number = 514 , length = 1264 refCount = 1, 448=data-448 number = 448 , length = 1412 refCount = 3, 57=data-57 number = 57 , length = 1431 refCount = 3, 468=data-468 number = 468 , length = 1266 refCount = 1, 200=data-200 number = 200 , length = 1431 refCount = 3, 163=data-163 number = 163 , length = 1264 refCount = 1, 632=data-632 number = 632 , length = 1653 refCount = 3, 617=data-617 number = 617 , length = 1264 refCount = 1, 579=data-579 number = 579 , length = 1264 refCount = 1, 674=data-674 number = 674 , length = 1264 refCount = 1, 489=data-489 number = 489 , length = 1324 refCount = 2, 388=data-388 number = 388 , length = 1264 refCount = 1, 424=data-424 number = 424 , length = 1655 refCount = 3, 873=data-873 number = 873 , length = 1264 refCount = 1, 449=data-449 number = 449 , length = 1264 refCount = 1, 518=data-518 number = 518 , length = 1266 refCount = 1, 639=data-639 number = 639 , length = 1352 refCount = 2, 383=data-383 number = 383 , length = 1264 refCount = 1, 127=data-127 number = 127 , length = 1264 refCount = 1, 1029=data-1029 number = 1029 , length = 2048 refCount = 7, 895=data-895 number = 895 , length = 1324 refCount = 2, 567=data-567 number = 567 , length = 1264 refCount = 1, 362=data-362 number = 362 , length = 1264 refCount = 1, 826=data-826 number = 826 , length = 1264 refCount = 1, 856=data-856 number = 856 , length = 1343 refCount = 2, 45=data-45 number = 45 , length = 1431 refCount = 3, 1071=data-1071 number = 1071 , length = 1261 refCount = 8, 1006=data-1006 number = 1006 , length = 1817 refCount = 8, 493=data-493 number = 493 , length = 1264 refCount = 1, 917=data-917 number = 917 , length = 1657 refCount = 3, 751=data-751 number = 751 , length = 1264 refCount = 1, 1033=data-1033 number = 1033 , length = 2039 refCount = 7, 812=data-812 number = 812 , length = 1264 refCount = 1, 240=data-240 number = 240 , length = 1264 refCount = 1, 743=data-743 number = 743 , length = 1345 refCount = 2, 398=data-398 number = 398 , length = 1264 refCount = 1, 526=data-526 number = 526 , length = 1324 refCount = 2, 188=data-188 number = 188 , length = 1324 refCount = 2, 297=data-297 number = 297 , length = 1264 refCount = 1, 70=data-70 number = 70 , length = 1264 refCount = 1, 411=data-411 number = 411 , length = 1266 refCount = 1, 106=data-106 number = 106 , length = 1716 refCount = 4, 587=data-587 number = 587 , length = 1264 refCount = 1, 1037=data-1037 number = 1037 , length = 1965 refCount = 10, 775=data-775 number = 775 , length = 1264 refCount = 1, 560=data-560 number = 560 , length = 1659 refCount = 3, 131=data-131 number = 131 , length = 1352 refCount = 2, 818=data-818 number = 818 , length = 1264 refCount = 1, 978=data-978 number = 978 , length = 1659 refCount = 3, 175=data-175 number = 175 , length = 1345 refCount = 2, 648=data-648 number = 648 , length = 1266 refCount = 1, 209=data-209 number = 209 , length = 1264 refCount = 1, 848=data-848 number = 848 , length = 1264 refCount = 1, 700=data-700 number = 700 , length = 1343 refCount = 2, 392=data-392 number = 392 , length = 1264 refCount = 1, 485=data-485 number = 485 , length = 1266 refCount = 1, 228=data-228 number = 228 , length = 1264 refCount = 1, 909=data-909 number = 909 , length = 1264 refCount = 1, 628=data-628 number = 628 , length = 1343 refCount = 2, 333=data-333 number = 333 , length = 1264 refCount = 1, 249=data-249 number = 249 , length = 1264 refCount = 1, 822=data-822 number = 822 , length = 1264 refCount = 1, 664=data-664 number = 664 , length = 1264 refCount = 1, 327=data-327 number = 327 , length = 1264 refCount = 1, 563=data-563 number = 563 , length = 1345 refCount = 2, 990=data-990 number = 990 , length = 2011 refCount = 7, 721=data-721 number = 721 , length = 1264 refCount = 1, 403=data-403 number = 403 , length = 1264 refCount = 1, 761=data-761 number = 761 , length = 1264 refCount = 1, 660=data-660 number = 660 , length = 1264 refCount = 1, 472=data-472 number = 472 , length = 1653 refCount = 3, 897=data-897 number = 897 , length = 1264 refCount = 1, 840=data-840 number = 840 , length = 1597 refCount = 2, 986=data-986 number = 986 , length = 1326 refCount = 2, 394=data-394 number = 394 , length = 1343 refCount = 2, 306=data-306 number = 306 , length = 1593 refCount = 2, 621=data-621 number = 621 , length = 1343 refCount = 2, 148=data-148 number = 148 , length = 1264 refCount = 1, 1054=data-1054 number = 1054 , length = 2039 refCount = 7, 765=data-765 number = 765 , length = 1266 refCount = 1, 1075=data-1075 number = 1075 , length = 2011 refCount = 7, 901=data-901 number = 901 , length = 1264 refCount = 1, 476=data-476 number = 476 , length = 1264 refCount = 1, 725=data-725 number = 725 , length = 1264 refCount = 1, 583=data-583 number = 583 , length = 1264 refCount = 1, 110=data-110 number = 110 , length = 1264 refCount = 1, 543=data-543 number = 543 , length = 1312 refCount = 1, 992=data-992 number = 992 , length = 2011 refCount = 7, 30=data-30 number = 30 , length = 1264 refCount = 1, 351=data-351 number = 351 , length = 1264 refCount = 1, 158=data-158 number = 158 , length = 1314 refCount = 1, 502=data-502 number = 502 , length = 1655 refCount = 3, 414=data-414 number = 414 , length = 1264 refCount = 1, 799=data-799 number = 799 , length = 1657 refCount = 3, 1048=data-1048 number = 1048 , length = 2011 refCount = 7, 736=data-736 number = 736 , length = 1264 refCount = 1, 32=data-32 number = 32 , length = 1266 refCount = 1, 657=data-657 number = 657 , length = 1264 refCount = 1, 1060=data-1060 number = 1060 , length = 1766 refCount = 7, 457=data-457 number = 457 , length = 1264 refCount = 1, 881=data-881 number = 881 , length = 1264 refCount = 1, 288=data-288 number = 288 , length = 1324 refCount = 2, 994=data-994 number = 994 , length = 1905 refCount = 9, 347=data-347 number = 347 , length = 1659 refCount = 3, 309=data-309 number = 309 , length = 1266 refCount = 1, 258=data-258 number = 258 , length = 1264 refCount = 1, 482=data-482 number = 482 , length = 1653 refCount = 3, 203=data-203 number = 203 , length = 1597 refCount = 2, 1052=data-1052 number = 1052 , length = 2039 refCount = 7, 1015=data-1015 number = 1015 , length = 2039 refCount = 7, 594=data-594 number = 594 , length = 1264 refCount = 1, 81=data-81 number = 81 , length = 1264 refCount = 1, 166=data-166 number = 166 , length = 1264 refCount = 1, 669=data-669 number = 669 , length = 1266 refCount = 1, 1064=data-1064 number = 1064 , length = 2011 refCount = 7, 925=data-925 number = 925 , length = 1264 refCount = 1, 376=data-376 number = 376 , length = 1264 refCount = 1, 973=data-973 number = 973 , length = 1659 refCount = 3, 835=data-835 number = 835 , length = 1264 refCount = 1, 262=data-262 number = 262 , length = 1324 refCount = 2, 717=data-717 number = 717 , length = 1266 refCount = 1, 548=data-548 number = 548 , length = 1593 refCount = 2, 1011=data-1011 number = 1011 , length = 2011 refCount = 7, 419=data-419 number = 419 , length = 1597 refCount = 2, 921=data-921 number = 921 , length = 2022 refCount = 5, 665=data-665 number = 665 , length = 1324 refCount = 2, 862=data-862 number = 862 , length = 1264 refCount = 1, 321=data-321 number = 321 , length = 1352 refCount = 2, 606=data-606 number = 606 , length = 1324 refCount = 2, 528=data-528 number = 528 , length = 1266 refCount = 1, 410=data-410 number = 410 , length = 1264 refCount = 1, 26=data-26 number = 26 , length = 1579 refCount = 5, 931=data-931 number = 931 , length = 1314 refCount = 1, 154=data-154 number = 154 , length = 1352 refCount = 2, 705=data-705 number = 705 , length = 1412 refCount = 3, 795=data-795 number = 795 , length = 1312 refCount = 1, 713=data-713 number = 713 , length = 1266 refCount = 1, 498=data-498 number = 498 , length = 1264 refCount = 1, 661=data-661 number = 661 , length = 1599 refCount = 2, 1072=data-1072 number = 1072 , length = 2011 refCount = 7, 453=data-453 number = 453 , length = 1266 refCount = 1, 709=data-709 number = 709 , length = 1264 refCount = 1, 935=data-935 number = 935 , length = 1326 refCount = 2, 1019=data-1019 number = 1019 , length = 2039 refCount = 7, 207=data-207 number = 207 , length = 1412 refCount = 3, 556=data-556 number = 556 , length = 1264 refCount = 1, 363=data-363 number = 363 , length = 1324 refCount = 2, 780=data-780 number = 780 , length = 1264 refCount = 1, 144=data-144 number = 144 , length = 1633 refCount = 3, 843=data-843 number = 843 , length = 1264 refCount = 1, 654=data-654 number = 654 , length = 1264 refCount = 1, 753=data-753 number = 753 , length = 1264 refCount = 1, 329=data-329 number = 329 , length = 1653 refCount = 3, 270=data-270 number = 270 , length = 1264 refCount = 1, 443=data-443 number = 443 , length = 1324 refCount = 2, 506=data-506 number = 506 , length = 1264 refCount = 1, 872=data-872 number = 872 , length = 1326 refCount = 2, 18=data-18 number = 18 , length = 1264 refCount = 1, 162=data-162 number = 162 , length = 1266 refCount = 1, 103=data-103 number = 103 , length = 1324 refCount = 2, 787=data-787 number = 787 , length = 1266 refCount = 1, 44=data-44 number = 44 , length = 1264 refCount = 1, 805=data-805 number = 805 , length = 1324 refCount = 2, 77=data-77 number = 77 , length = 1264 refCount = 1, 678=data-678 number = 678 , length = 1655 refCount = 3, 956=data-956 number = 956 , length = 1659 refCount = 3, 293=data-293 number = 293 , length = 1264 refCount = 1, 591=data-591 number = 591 , length = 1352 refCount = 2, 99=data-99 number = 99 , length = 1266 refCount = 1, 791=data-791 number = 791 , length = 1352 refCount = 2, 868=data-868 number = 868 , length = 1264 refCount = 1, 535=data-535 number = 535 , length = 1326 refCount = 2, 360=data-360 number = 360 , length = 1352 refCount = 2, 339=data-339 number = 339 , length = 1264 refCount = 1, 952=data-952 number = 952 , length = 1683 refCount = 3, 221=data-221 number = 221 , length = 1312 refCount = 1, 1068=data-1068 number = 1068 , length = 928 refCount = 7, 1023=data-1023 number = 1023 , length = 2039 refCount = 7, 1025=data-1025 number = 1025 , length = 2011 refCount = 7, 426=data-426 number = 426 , length = 1264 refCount = 1, 695=data-695 number = 695 , length = 1599 refCount = 2, 636=data-636 number = 636 , length = 1264 refCount = 1, 858=data-858 number = 858 , length = 1264 refCount = 1, 552=data-552 number = 552 , length = 1264 refCount = 1, 367=data-367 number = 367 , length = 1343 refCount = 2, 732=data-732 number = 732 , length = 1264 refCount = 1, 430=data-430 number = 430 , length = 1266 refCount = 1, 266=data-266 number = 266 , length = 1264 refCount = 1, 948=data-948 number = 948 , length = 1314 refCount = 1, 650=data-650 number = 650 , length = 1312 refCount = 1, 439=data-439 number = 439 , length = 1354 refCount = 2, 877=data-877 number = 877 , length = 1264 refCount = 1, 276=data-276 number = 276 , length = 1264 refCount = 1, 469=data-469 number = 469 , length = 1431 refCount = 3, 246=data-246 number = 246 , length = 1264 refCount = 1, 896=data-896 number = 896 , length = 1266 refCount = 1, 758=data-758 number = 758 , length = 1264 refCount = 1, 313=data-313 number = 313 , length = 1264 refCount = 1, 738=data-738 number = 738 , length = 1264 refCount = 1, 225=data-225 number = 225 , length = 1264 refCount = 1, 124=data-124 number = 124 , length = 1352 refCount = 2, 317=data-317 number = 317 , length = 1264 refCount = 1, 825=data-825 number = 825 , length = 1324 refCount = 2, 380=data-380 number = 380 , length = 1352 refCount = 2, 60=data-60 number = 60 , length = 1657 refCount = 3, 61=data-61 number = 61 , length = 1264 refCount = 1, 683=data-683 number = 683 , length = 1264 refCount = 1, 447=data-447 number = 447 , length = 1593 refCount = 2, 914=data-914 number = 914 , length = 1431 refCount = 3, 119=data-119 number = 119 , length = 1352 refCount = 2, 703=data-703 number = 703 , length = 1264 refCount = 1, 690=data-690 number = 690 , length = 1264 refCount = 1, 959=data-959 number = 959 , length = 1391 refCount = 2, 631=data-631 number = 631 , length = 1264 refCount = 1, 572=data-572 number = 572 , length = 1264 refCount = 1, 402=data-402 number = 402 , length = 1264 refCount = 1, 792=data-792 number = 792 , length = 1264 refCount = 1, 918=data-918 number = 918 , length = 1264 refCount = 1, 801=data-801 number = 801 , length = 1266 refCount = 1, 136=data-136 number = 136 , length = 1264 refCount = 1, 568=data-568 number = 568 , length = 1343 refCount = 2, 627=data-627 number = 627 , length = 1266 refCount = 1, 855=data-855 number = 855 , length = 1264 refCount = 1, 494=data-494 number = 494 , length = 1264 refCount = 1, 406=data-406 number = 406 , length = 1312 refCount = 1, 1007=data-1007 number = 1007 , length = 1881 refCount = 5, 694=data-694 number = 694 , length = 1264 refCount = 1, 515=data-515 number = 515 , length = 1266 refCount = 1, 65=data-65 number = 65 , length = 1653 refCount = 3, 746=data-746 number = 746 , length = 1264 refCount = 1, 452=data-452 number = 452 , length = 1312 refCount = 1, 280=data-280 number = 280 , length = 1264 refCount = 1, 242=data-242 number = 242 , length = 1264 refCount = 1, 961=data-961 number = 961 , length = 1766 refCount = 4, 813=data-813 number = 813 , length = 1659 refCount = 3, 1003=data-1003 number = 1003 , length = 2039 refCount = 7, 712=data-712 number = 712 , length = 1324 refCount = 2, 557=data-557 number = 557 , length = 1354 refCount = 2, 490=data-490 number = 490 , length = 1312 refCount = 1, 936=data-936 number = 936 , length = 1312 refCount = 1, 423=data-423 number = 423 , length = 1264 refCount = 1, 965=data-965 number = 965 , length = 1433 refCount = 3, 750=data-750 number = 750 , length = 1264 refCount = 1, 343=data-343 number = 343 , length = 1264 refCount = 1, 237=data-237 number = 237 , length = 1266 refCount = 1, 1028=data-1028 number = 1028 , length = 2011 refCount = 7, 56=data-56 number = 56 , length = 1264 refCount = 1, 486=data-486 number = 486 , length = 1352 refCount = 2, 464=data-464 number = 464 , length = 1264 refCount = 1, 326=data-326 number = 326 , length = 1431 refCount = 3, 999=data-999 number = 999 , length = 2011 refCount = 7, 620=data-620 number = 620 , length = 1312 refCount = 1, 699=data-699 number = 699 , length = 1264 refCount = 1, 48=data-48 number = 48 , length = 1324 refCount = 2, 187=data-187 number = 187 , length = 1352 refCount = 2, 73=data-73 number = 73 , length = 1314 refCount = 1, 985=data-985 number = 985 , length = 1821 refCount = 4, 910=data-910 number = 910 , length = 1324 refCount = 2, 385=data-385 number = 385 , length = 1264 refCount = 1, 582=data-582 number = 582 , length = 1264 refCount = 1, 523=data-523 number = 523 , length = 1352 refCount = 2, 809=data-809 number = 809 , length = 1264 refCount = 1, 322=data-322 number = 322 , length = 1266 refCount = 1, 300=data-300 number = 300 , length = 1264 refCount = 1, 465=data-465 number = 465 , length = 1324 refCount = 2, 250=data-250 number = 250 , length = 1264 refCount = 1, 776=data-776 number = 776 , length = 1264 refCount = 1, 884=data-884 number = 884 , length = 1264 refCount = 1, 742=data-742 number = 742 , length = 1352 refCount = 2, 578=data-578 number = 578 , length = 1264 refCount = 1, 519=data-519 number = 519 , length = 1324 refCount = 2, 847=data-847 number = 847 , length = 1391 refCount = 2, 229=data-229 number = 229 , length = 1264 refCount = 1, 687=data-687 number = 687 , length = 1264 refCount = 1, 397=data-397 number = 397 , length = 1412 refCount = 3, 898=data-898 number = 898 , length = 1391 refCount = 2, 1036=data-1036 number = 1036 , length = 2011 refCount = 7, 254=data-254 number = 254 , length = 1433 refCount = 3, 728=data-728 number = 728 , length = 1264 refCount = 1, 305=data-305 number = 305 , length = 1264 refCount = 1, 477=data-477 number = 477 , length = 1266 refCount = 1, 334=data-334 number = 334 , length = 1324 refCount = 2, 216=data-216 number = 216 , length = 1343 refCount = 2, 902=data-902 number = 902 , length = 1266 refCount = 1, 817=data-817 number = 817 , length = 1264 refCount = 1, 52=data-52 number = 52 , length = 1266 refCount = 1, 977=data-977 number = 977 , length = 1354 refCount = 2, 170=data-170 number = 170 , length = 1266 refCount = 1, 111=data-111 number = 111 , length = 1343 refCount = 2, 839=data-839 number = 839 , length = 1324 refCount = 2, 624=data-624 number = 624 , length = 1264 refCount = 1, 564=data-564 number = 564 , length = 1264 refCount = 1, 1076=data-1076 number = 1076 , length = 1766 refCount = 7, 473=data-473 number = 473 , length = 1264 refCount = 1, 641=data-641 number = 641 , length = 1264 refCount = 1, 906=data-906 number = 906 , length = 1266 refCount = 1, 981=data-981 number = 981 , length = 1391 refCount = 2, 766=data-766 number = 766 , length = 1653 refCount = 3, 602=data-602 number = 602 , length = 1676 refCount = 3, 389=data-389 number = 389 , length = 1266 refCount = 1, 829=data-829 number = 829 , length = 1264 refCount = 1, 892=data-892 number = 892 , length = 1593 refCount = 2, 69=data-69 number = 69 , length = 1354 refCount = 2, 972=data-972 number = 972 , length = 1400 refCount = 2, 886=data-886 number = 886 , length = 1657 refCount = 3, 735=data-735 number = 735 , length = 1264 refCount = 1, 800=data-800 number = 800 , length = 1264 refCount = 1, 716=data-716 number = 716 , length = 1312 refCount = 1, 287=data-287 number = 287 , length = 1266 refCount = 1, 214=data-214 number = 214 , length = 1324 refCount = 2, 991=data-991 number = 991 , length = 2048 refCount = 7, 907=data-907 number = 907 , length = 1431 refCount = 3, 544=data-544 number = 544 , length = 1491 refCount = 4, 31=data-31 number = 31 , length = 1593 refCount = 2, 993=data-993 number = 993 , length = 2020 refCount = 7, 413=data-413 number = 413 , length = 1393 refCount = 2, 926=data-926 number = 926 , length = 1326 refCount = 2, 96=data-96 number = 96 , length = 1266 refCount = 1, 857=data-857 number = 857 , length = 1354 refCount = 2, 1057=data-1057 number = 1057 , length = 1766 refCount = 7, 157=data-157 number = 157 , length = 1264 refCount = 1, 651=data-651 number = 651 , length = 1264 refCount = 1, 670=data-670 number = 670 , length = 1264 refCount = 1, 352=data-352 number = 352 , length = 1681 refCount = 3, 100=data-100 number = 100 , length = 1431 refCount = 3, 796=data-796 number = 796 , length = 1431 refCount = 3, 283=data-283 number = 283 , length = 1264 refCount = 1, 373=data-373 number = 373 , length = 1264 refCount = 1, 194=data-194 number = 194 , length = 1352 refCount = 2, 540=data-540 number = 540 , length = 1264 refCount = 1, 259=data-259 number = 259 , length = 1352 refCount = 2, 27=data-27 number = 27 , length = 1264 refCount = 1, 605=data-605 number = 605 , length = 1264 refCount = 1, 503=data-503 number = 503 , length = 1264 refCount = 1, 1012=data-1012 number = 1012 , length = 2039 refCount = 7, 1051=data-1051 number = 1051 , length = 2020 refCount = 7, 821=data-821 number = 821 , length = 1312 refCount = 1, 861=data-861 number = 861 , length = 1264 refCount = 1, 708=data-708 number = 708 , length = 1676 refCount = 3, 536=data-536 number = 536 , length = 1264 refCount = 1, 643=data-643 number = 643 , length = 1312 refCount = 1, 35=data-35 number = 35 , length = 1266 refCount = 1, 932=data-932 number = 932 , length = 1264 refCount = 1, 462=data-462 number = 462 , length = 1352 refCount = 2, 545=data-545 number = 545 , length = 1264 refCount = 1, 149=data-149 number = 149 , length = 1493 refCount = 4, 662=data-662 number = 662 , length = 1352 refCount = 2, 499=data-499 number = 499 , length = 1433 refCount = 3, 324=data-324 number = 324 , length = 1597 refCount = 2, 206=data-206 number = 206 , length = 1264 refCount = 1, 88=data-88 number = 88 , length = 1433 refCount = 3, 364=data-364 number = 364 , length = 1264 refCount = 1, 409=data-409 number = 409 , length = 1412 refCount = 3, 968=data-968 number = 968 , length = 1354 refCount = 2, 458=data-458 number = 458 , length = 1264 refCount = 1, 922=data-922 number = 922 , length = 1953 refCount = 4, 108=data-108 number = 108 , length = 1324 refCount = 2, 773=data-773 number = 773 , length = 1264 refCount = 1, 586=data-586 number = 586 , length = 1264 refCount = 1, 679=data-679 number = 679 , length = 1264 refCount = 1, 842=data-842 number = 842 , length = 1431 refCount = 3, 312=data-312 number = 312 , length = 1266 refCount = 1, 153=data-153 number = 153 , length = 1266 refCount = 1, 597=data-597 number = 597 , length = 1264 refCount = 1, 202=data-202 number = 202 , length = 1266 refCount = 1, 666=data-666 number = 666 , length = 1312 refCount = 1, 1069=data-1069 number = 1069 , length = 2011 refCount = 7, 454=data-454 number = 454 , length = 1264 refCount = 1, 871=data-871 number = 871 , length = 1264 refCount = 1, 955=data-955 number = 955 , length = 1400 refCount = 2, 368=data-368 number = 368 , length = 1264 refCount = 1, 553=data-553 number = 553 , length = 1264 refCount = 1, 198=data-198 number = 198 , length = 1264 refCount = 1, 271=data-271 number = 271 , length = 1264 refCount = 1, 610=data-610 number = 610 , length = 1431 refCount = 3, 507=data-507 number = 507 , length = 1264 refCount = 1, 353=data-353 number = 353 , length = 1264 refCount = 1, 777=data-777 number = 777 , length = 1266 refCount = 1, 442=data-442 number = 442 , length = 1599 refCount = 2, 867=data-867 number = 867 , length = 1264 refCount = 1, 549=data-549 number = 549 , length = 1264 refCount = 1, 731=data-731 number = 731 , length = 1264 refCount = 1, 303=data-303 number = 303 , length = 1431 refCount = 3, 806=data-806 number = 806 , length = 1264 refCount = 1, 340=data-340 number = 340 , length = 1326 refCount = 2, 614=data-614 number = 614 , length = 1264 refCount = 1, 222=data-222 number = 222 , length = 1264 refCount = 1, 691=data-691 number = 691 , length = 1264 refCount = 1, 422=data-422 number = 422 , length = 1264 refCount = 1, 438=data-438 number = 438 , length = 1324 refCount = 2, 943=data-943 number = 943 , length = 1822 refCount = 5, 947=data-947 number = 947 , length = 1326 refCount = 2, 296=data-296 number = 296 , length = 1264 refCount = 1, 590=data-590 number = 590 , length = 1264 refCount = 1, 593=data-593 number = 593 , length = 1264 refCount = 1, 788=data-788 number = 788 , length = 1264 refCount = 1, 849=data-849 number = 849 , length = 1264 refCount = 1, 178=data-178 number = 178 , length = 1264 refCount = 1, 1065=data-1065 number = 1065 , length = 1766 refCount = 7, 532=data-532 number = 532 , length = 1685 refCount = 3, 218=data-218 number = 218 , length = 1264 refCount = 1, 951=data-951 number = 951 , length = 1433 refCount = 3, 267=data-267 number = 267 , length = 1264 refCount = 1, 769=data-769 number = 769 , length = 1266 refCount = 1, 1043=data-1043 number = 1043 , length = 1960 refCount = 6, 434=data-434 number = 434 , length = 1264 refCount = 1, 39=data-39 number = 39 , length = 1264 refCount = 1, 573=data-573 number = 573 , length = 1264 refCount = 1, 1026=data-1026 number = 1026 , length = 1766 refCount = 7, 429=data-429 number = 429 , length = 1264 refCount = 1, 1020=data-1020 number = 1020 , length = 2011 refCount = 7, 348=data-348 number = 348 , length = 1264 refCount = 1, 878=data-878 number = 878 , length = 1266 refCount = 1, 655=data-655 number = 655 , length = 1266 refCount = 1, 511=data-511 number = 511 , length = 1264 refCount = 1, 491=data-491 number = 491 , length = 1343 refCount = 2, 1004=data-1004 number = 1004 , length = 2020 refCount = 7, 854=data-854 number = 854 , length = 1314 refCount = 1, 190=data-190 number = 190 , length = 1264 refCount = 1, 832=data-832 number = 832 , length = 1264 refCount = 1, 684=data-684 number = 684 , length = 1659 refCount = 3, 446=data-446 number = 446 , length = 1343 refCount = 2, 576=data-576 number = 576 , length = 1264 refCount = 1, 470=data-470 number = 470 , length = 1264 refCount = 1, 737=data-737 number = 737 , length = 1433 refCount = 3, 958=data-958 number = 958 , length = 1326 refCount = 2, 59=data-59 number = 59 , length = 1264 refCount = 1, 569=data-569 number = 569 , length = 1264 refCount = 1, 169=data-169 number = 169 , length = 1264 refCount = 1, 316=data-316 number = 316 , length = 1264 refCount = 1, 234=data-234 number = 234 , length = 1264 refCount = 1, 68=data-68 number = 68 , length = 1343 refCount = 2, 828=data-828 number = 828 , length = 1431 refCount = 3, 401=data-401 number = 401 , length = 1352 refCount = 2, 759=data-759 number = 759 , length = 1354 refCount = 2, 939=data-939 number = 939 , length = 1264 refCount = 1, 915=data-915 number = 915 , length = 1266 refCount = 1, 626=data-626 number = 626 , length = 1312 refCount = 1, 181=data-181 number = 181 , length = 1264 refCount = 1, 279=data-279 number = 279 , length = 1264 refCount = 1, 356=data-356 number = 356 , length = 1264 refCount = 1, 565=data-565 number = 565 , length = 1264 refCount = 1, 320=data-320 number = 320 , length = 1264 refCount = 1, 247=data-247 number = 247 , length = 1264 refCount = 1, 120=data-120 number = 120 , length = 1324 refCount = 2, 676=data-676 number = 676 , length = 1264 refCount = 1, 405=data-405 number = 405 , length = 1264 refCount = 1, 516=data-516 number = 516 , length = 1431 refCount = 3, 919=data-919 number = 919 , length = 1431 refCount = 3, 291=data-291 number = 291 , length = 1354 refCount = 2, 745=data-745 number = 745 , length = 1681 refCount = 3, 161=data-161 number = 161 , length = 1352 refCount = 2, 1027=data-1027 number = 1027 , length = 2039 refCount = 7, 381=data-381 number = 381 , length = 1266 refCount = 1, 116=data-116 number = 116 , length = 1593 refCount = 2, 802=data-802 number = 802 , length = 1431 refCount = 3, 344=data-344 number = 344 , length = 1433 refCount = 3, 698=data-698 number = 698 , length = 1266 refCount = 1, 495=data-495 number = 495 , length = 1326 refCount = 2, 1008=data-1008 number = 1008 , length = 2011 refCount = 7, 749=data-749 number = 749 , length = 1312 refCount = 1, 451=data-451 number = 451 , length = 1343 refCount = 2, 76=data-76 number = 76 , length = 1264 refCount = 1, 875=data-875 number = 875 , length = 1345 refCount = 2, 711=data-711 number = 711 , length = 1264 refCount = 1, 577=data-577 number = 577 , length = 1264 refCount = 1, 558=data-558 number = 558 , length = 1264 refCount = 1, 637=data-637 number = 637 , length = 1266 refCount = 1, 238=data-238 number = 238 , length = 1324 refCount = 2, 702=data-702 number = 702 , length = 1312 refCount = 1, 814=data-814 number = 814 , length = 1264 refCount = 1, 893=data-893 number = 893 , length = 1264 refCount = 1, 15=data-15 number = 15 , length = 1352 refCount = 2, 400=data-400 number = 400 , length = 1672 refCount = 3, 763=data-763 number = 763 , length = 1431 refCount = 3, 581=data-581 number = 581 , length = 1264 refCount = 1, 524=data-524 number = 524 , length = 1264 refCount = 1, 112=data-112 number = 112 , length = 1266 refCount = 1, 299=data-299 number = 299 , length = 1264 refCount = 1, 230=data-230 number = 230 , length = 1433 refCount = 3, 1035=data-1035 number = 1035 , length = 2048 refCount = 7, 129=data-129 number = 129 , length = 1266 refCount = 1, 810=data-810 number = 810 , length = 1354 refCount = 2, 988=data-988 number = 988 , length = 1414 refCount = 3, 838=data-838 number = 838 , length = 1266 refCount = 1, 619=data-619 number = 619 , length = 1324 refCount = 2, 47=data-47 number = 47 , length = 1266 refCount = 1, 186=data-186 number = 186 , length = 1266 refCount = 1, 976=data-976 number = 976 , length = 1391 refCount = 2, 903=data-903 number = 903 , length = 1324 refCount = 2, 688=data-688 number = 688 , length = 1264 refCount = 1, 386=data-386 number = 386 , length = 1354 refCount = 2, 487=data-487 number = 487 , length = 1264 refCount = 1, 1000=data-1000 number = 1000 , length = 1766 refCount = 7, 520=data-520 number = 520 , length = 1312 refCount = 1, 466=data-466 number = 466 , length = 1314 refCount = 1, 264=data-264 number = 264 , length = 1343 refCount = 2, 642=data-642 number = 642 , length = 1324 refCount = 2, 461=data-461 number = 461 , length = 1264 refCount = 1, 834=data-834 number = 834 , length = 1343 refCount = 2, 1073=data-1073 number = 1073 , length = 1766 refCount = 7, 741=data-741 number = 741 , length = 1264 refCount = 1, 1031=data-1031 number = 1031 , length = 2020 refCount = 7, 1056=data-1056 number = 1056 , length = 2011 refCount = 7, 51=data-51 number = 51 , length = 1431 refCount = 3, 1077=data-1077 number = 1077 , length = 841 refCount = 5, 478=data-478 number = 478 , length = 1597 refCount = 2, 390=data-390 number = 390 , length = 1312 refCount = 1, 727=data-727 number = 727 , length = 1266 refCount = 1, 483=data-483 number = 483 , length = 1264 refCount = 1, 996=data-996 number = 996 , length = 2039 refCount = 7, 377=data-377 number = 377 , length = 1264 refCount = 1, 980=data-980 number = 980 , length = 1326 refCount = 2, 889=data-889 number = 889 , length = 1431 refCount = 3, 561=data-561 number = 561 , length = 1264 refCount = 1, 646=data-646 number = 646 , length = 1431 refCount = 3, 211=data-211 number = 211 , length = 1352 refCount = 2, 308=data-308 number = 308 , length = 1264 refCount = 1, 846=data-846 number = 846 , length = 1264 refCount = 1, 723=data-723 number = 723 , length = 1264 refCount = 1, 767=data-767 number = 767 , length = 1264 refCount = 1, 474=data-474 number = 474 , length = 1266 refCount = 1, 255=data-255 number = 255 , length = 1264 refCount = 1, 91=data-91 number = 91 , length = 1593 refCount = 2, 984=data-984 number = 984 , length = 1266 refCount = 1, 173=data-173 number = 173 , length = 1343 refCount = 2, 820=data-820 number = 820 , length = 1264 refCount = 1, 133=data-133 number = 133 , length = 1264 refCount = 1, 623=data-623 number = 623 , length = 1266 refCount = 1, 325=data-325 number = 325 , length = 1266 refCount = 1, 908=data-908 number = 908 , length = 1266 refCount = 1, 652=data-652 number = 652 , length = 1266 refCount = 1, 671=data-671 number = 671 , length = 1324 refCount = 2, 864=data-864 number = 864 , length = 1264 refCount = 1, 971=data-971 number = 971 , length = 1266 refCount = 1, 927=data-927 number = 927 , length = 1312 refCount = 1, 374=data-374 number = 374 , length = 1264 refCount = 1, 608=data-608 number = 608 , length = 1264 refCount = 1, 223=data-223 number = 223 , length = 1264 refCount = 1, 726=data-726 number = 726 , length = 1324 refCount = 2, 286=data-286 number = 286 , length = 1264 refCount = 1, 34=data-34 number = 34 , length = 1352 refCount = 2, 1050=data-1050 number = 1050 , length = 2039 refCount = 7, 160=data-160 number = 160 , length = 1345 refCount = 2, 396=data-396 number = 396 , length = 1599 refCount = 2, 416=data-416 number = 416 , length = 1352 refCount = 2, 715=data-715 number = 715 , length = 1264 refCount = 1, 659=data-659 number = 659 , length = 1343 refCount = 2, 459=data-459 number = 459 , length = 1312 refCount = 1, 437=data-437 number = 437 , length = 1264 refCount = 1, 146=data-146 number = 146 , length = 1324 refCount = 2, 946=data-946 number = 946 , length = 1433 refCount = 3, 887=data-887 number = 887 , length = 1264 refCount = 1, 541=data-541 number = 541 , length = 1412 refCount = 3, 38=data-38 number = 38 , length = 1266 refCount = 1, 28=data-28 number = 28 , length = 1264 refCount = 1, 1013=data-1013 number = 1013 , length = 1965 refCount = 10, 770=data-770 number = 770 , length = 1264 refCount = 1, 504=data-504 number = 504 , length = 1264 refCount = 1, 1046=data-1046 number = 1046 , length = 1766 refCount = 7, 797=data-797 number = 797 , length = 1266 refCount = 1, 479=data-479 number = 479 , length = 1352 refCount = 2, 644=data-644 number = 644 , length = 1264 refCount = 1, 168=data-168 number = 168 , length = 1352 refCount = 2, 929=data-929 number = 929 , length = 1264 refCount = 1, 87=data-87 number = 87 , length = 1264 refCount = 1, 600=data-600 number = 600 , length = 1264 refCount = 1, 663=data-663 number = 663 , length = 1266 refCount = 1, 883=data-883 number = 883 , length = 1431 refCount = 3, 455=data-455 number = 455 , length = 1352 refCount = 2, 824=data-824 number = 824 , length = 1354 refCount = 2, 537=data-537 number = 537 , length = 1264 refCount = 1, 193=data-193 number = 193 , length = 1266 refCount = 1, 349=data-349 number = 349 , length = 1352 refCount = 2, 260=data-260 number = 260 , length = 1264 refCount = 1, 546=data-546 number = 546 , length = 1764 refCount = 4, 774=data-774 number = 774 , length = 1433 refCount = 3, 93=data-93 number = 93 , length = 1345 refCount = 2, 841=data-841 number = 841 , length = 1266 refCount = 1, 417=data-417 number = 417 , length = 1264 refCount = 1, 24=data-24 number = 24 , length = 1264 refCount = 1, 500=data-500 number = 500 , length = 1264 refCount = 1, 967=data-967 number = 967 , length = 1391 refCount = 2, 833=data-833 number = 833 , length = 1312 refCount = 1, 667=data-667 number = 667 , length = 1264 refCount = 1, 1062=data-1062 number = 1062 , length = 2039 refCount = 7, 1038=data-1038 number = 1038 , length = 2011 refCount = 7, 585=data-585 number = 585 , length = 1264 refCount = 1, 370=data-370 number = 370 , length = 1264 refCount = 1, 282=data-282 number = 282 , length = 1264 refCount = 1, 680=data-680 number = 680 , length = 1264 refCount = 1, 323=data-323 number = 323 , length = 1324 refCount = 2, 302=data-302 number = 302 , length = 1264 refCount = 1, 46=data-46 number = 46 , length = 1264 refCount = 1, 1009=data-1009 number = 1009 , length = 2039 refCount = 7, 807=data-807 number = 807 , length = 1312 refCount = 1, 432=data-432 number = 432 , length = 1433 refCount = 3, 441=data-441 number = 441 , length = 1264 refCount = 1, 609=data-609 number = 609 , length = 1312 refCount = 1, 508=data-508 number = 508 , length = 1599 refCount = 2, 97=data-97 number = 97 , length = 1324 refCount = 2, 529=data-529 number = 529 , length = 1431 refCount = 3, 866=data-866 number = 866 , length = 1595 refCount = 2, 785=data-785 number = 785 , length = 1264 refCount = 1, 782=data-782 number = 782 , length = 1264 refCount = 1, 142=data-142 number = 142 , length = 1343 refCount = 2, 354=data-354 number = 354 , length = 1264 refCount = 1, 83=data-83 number = 83 , length = 1264 refCount = 1, 596=data-596 number = 596 , length = 1312 refCount = 1, 268=data-268 number = 268 , length = 1326 refCount = 2, 613=data-613 number = 613 , length = 1657 refCount = 3, 331=data-331 number = 331 , length = 1431 refCount = 3, 1070=data-1070 number = 1070 , length = 1905 refCount = 9, 816=data-816 number = 816 , length = 1433 refCount = 3, 274=data-274 number = 274 , length = 1264 refCount = 1, 933=data-933 number = 933 , length = 1433 refCount = 3, 870=data-870 number = 870 , length = 1352 refCount = 2, 421=data-421 number = 421 , length = 1433 refCount = 3, 719=data-719 number = 719 , length = 1412 refCount = 3, 550=data-550 number = 550 , length = 1352 refCount = 2, 433=data-433 number = 433 , length = 1264 refCount = 1, 177=data-177 number = 177 , length = 1314 refCount = 1, 589=data-589 number = 589 , length = 1264 refCount = 1, 1066=data-1066 number = 1066 , length = 841 refCount = 5, 633=data-633 number = 633 , length = 1264 refCount = 1, 1017=data-1017 number = 1017 , length = 2039 refCount = 7, 852=data-852 number = 852 , length = 1324 refCount = 2, 734=data-734 number = 734 , length = 1314 refCount = 1, 42=data-42 number = 42 , length = 1266 refCount = 1, 760=data-760 number = 760 , length = 1324 refCount = 2, 860=data-860 number = 860 , length = 1659 refCount = 3, 656=data-656 number = 656 , length = 1653 refCount = 3, 295=data-295 number = 295 , length = 1312 refCount = 1, 345=data-345 number = 345 , length = 1264 refCount = 1, 219=data-219 number = 219 , length = 1264 refCount = 1, 513=data-513 number = 513 , length = 1312 refCount = 1, 879=data-879 number = 879 , length = 1324 refCount = 2, 1021=data-1021 number = 1021 , length = 1965 refCount = 10, 778=data-778 number = 778 , length = 1372 refCount = 2, 942=data-942 number = 942 , length = 1683 refCount = 3, 512=data-512 number = 512 , length = 1324 refCount = 2, 789=data-789 number = 789 , length = 1672 refCount = 3, 20=data-20 number = 20 , length = 1264 refCount = 1, 533=data-533 number = 533 , length = 1264 refCount = 1, 730=data-730 number = 730 , length = 1431 refCount = 3, 365=data-365 number = 365 , length = 1266 refCount = 1, 341=data-341 number = 341 , length = 1312 refCount = 1, 315=data-315 number = 315 , length = 1266 refCount = 1, 235=data-235 number = 235 , length = 1352 refCount = 2, 748=data-748 number = 748 , length = 1266 refCount = 1, 827=data-827 number = 827 , length = 1599 refCount = 2, 768=data-768 number = 768 , length = 1431 refCount = 3, 681=data-681 number = 681 , length = 1433 refCount = 3, 113=data-113 number = 113 , length = 1633 refCount = 3, 425=data-425 number = 425 , length = 1264 refCount = 1, 938=data-938 number = 938 , length = 1433 refCount = 3, 625=data-625 number = 625 , length = 1324 refCount = 2, 117=data-117 number = 117 , length = 1264 refCount = 1, 634=data-634 number = 634 , length = 1266 refCount = 1, 189=data-189 number = 189 , length = 1266 refCount = 1, 122=data-122 number = 122 , length = 1393 refCount = 2, 575=data-575 number = 575 , length = 1264 refCount = 1, 450=data-450 number = 450 , length = 1266 refCount = 1, 63=data-63 number = 63 , length = 1431 refCount = 3, 831=data-831 number = 831 , length = 1324 refCount = 2, 172=data-172 number = 172 , length = 1354 refCount = 2, 248=data-248 number = 248 , length = 1433 refCount = 3, 756=data-756 number = 756 , length = 1312 refCount = 1, 629=data-629 number = 629 , length = 1352 refCount = 2, 126=data-126 number = 126 , length = 1324 refCount = 2, 963=data-963 number = 963 , length = 1400 refCount = 2, 517=data-517 number = 517 , length = 1264 refCount = 1, 673=data-673 number = 673 , length = 1343 refCount = 2, 580=data-580 number = 580 , length = 1264 refCount = 1, 900=data-900 number = 900 , length = 1352 refCount = 2, 382=data-382 number = 382 , length = 1264 refCount = 1, 54=data-54 number = 54 , length = 1324 refCount = 2, 618=data-618 number = 618 , length = 1266 refCount = 1, 488=data-488 number = 488 , length = 1266 refCount = 1, 874=data-874 number = 874 , length = 1312 refCount = 1, 361=data-361 number = 361 , length = 1266 refCount = 1, 570=data-570 number = 570 , length = 1264 refCount = 1, 1001=data-1001 number = 1001 , length = 1960 refCount = 6, 803=data-803 number = 803 , length = 1264 refCount = 1, 67=data-67 number = 67 , length = 1264 refCount = 1, 701=data-701 number = 701 , length = 1264 refCount = 1, 197=data-197 number = 197 , length = 1653 refCount = 3, 957=data-957 number = 957 , length = 1433 refCount = 3, 894=data-894 number = 894 , length = 1352 refCount = 2, 164=data-164 number = 164 , length = 1324 refCount = 2, 853=data-853 number = 853 , length = 1266 refCount = 1, 638=data-638 number = 638 , length = 1597 refCount = 2, 1005=data-1005 number = 1005 , length = 2039 refCount = 7, 588=data-588 number = 588 , length = 1264 refCount = 1, 336=data-336 number = 336 , length = 1343 refCount = 2, 525=data-525 number = 525 , length = 1266 refCount = 1, 1034=data-1034 number = 1034 , length = 2011 refCount = 7, 399=data-399 number = 399 , length = 1266 refCount = 1, 176=data-176 number = 176 , length = 1264 refCount = 1, 697=data-697 number = 697 , length = 1412 refCount = 3, 252=data-252 number = 252 , length = 1264 refCount = 1, 924=data-924 number = 924 , length = 1354 refCount = 2, 475=data-475 number = 475 , length = 1431 refCount = 3, 987=data-987 number = 987 , length = 1391 refCount = 2, 1030=data-1030 number = 1030 , length = 2011 refCount = 7, 75=data-75 number = 75 , length = 1354 refCount = 2, 231=data-231 number = 231 , length = 1264 refCount = 1, 744=data-744 number = 744 , length = 1264 refCount = 1, 722=data-722 number = 722 , length = 1676 refCount = 3, 130=data-130 number = 130 , length = 1343 refCount = 2, 811=data-811 number = 811 , length = 1264 refCount = 1, 387=data-387 number = 387 , length = 1343 refCount = 2, 12=data-12 number = 12 , length = 1264 refCount = 1, 975=data-975 number = 975 , length = 1326 refCount = 2, 904=data-904 number = 904 , length = 1314 refCount = 1, 562=data-562 number = 562 , length = 1264 refCount = 1, 412=data-412 number = 412 , length = 1264 refCount = 1, 997=data-997 number = 997 , length = 2011 refCount = 7, 647=data-647 number = 647 , length = 1264 refCount = 1, 566=data-566 number = 566 , length = 1324 refCount = 2, 1074=data-1074 number = 1074 , length = 1261 refCount = 8, 559=data-559 number = 559 , length = 1264 refCount = 1, 185=data-185 number = 185 , length = 1264 refCount = 1, 227=data-227 number = 227 , length = 1659 refCount = 3, 151=data-151 number = 151 , length = 1264 refCount = 1, 8=data-8 number = 8 , length = 1343 refCount = 2, 845=data-845 number = 845 , length = 1324 refCount = 2, 391=data-391 number = 391 , length = 1324 refCount = 2, 298=data-298 number = 298 , length = 1433 refCount = 3, 134=data-134 number = 134 , length = 1324 refCount = 2, 916=data-916 number = 916 , length = 1264 refCount = 1, 467=data-467 number = 467 , length = 1264 refCount = 1, 307=data-307 number = 307 , length = 1352 refCount = 2, 147=data-147 number = 147 , length = 1597 refCount = 2, 408=data-408 number = 408 , length = 1266 refCount = 1, 395=data-395 number = 395 , length = 1264 refCount = 1, 50=data-50 number = 50 , length = 1266 refCount = 1, 912=data-912 number = 912 , length = 1266 refCount = 1, 685=data-685 number = 685 , length = 1352 refCount = 2, 521=data-521 number = 521 , length = 1264 refCount = 1, 890=data-890 number = 890 , length = 1264 refCount = 1, 979=data-979 number = 979 , length = 1433 refCount = 3, 764=data-764 number = 764 , length = 1264 refCount = 1, 604=data-604 number = 604 , length = 1354 refCount = 2, 584=data-584 number = 584 , length = 1264 refCount = 1, 837=data-837 number = 837 , length = 1264 refCount = 1, 622=data-622 number = 622 , length = 1352 refCount = 2, 328=data-328 number = 328 , length = 1266 refCount = 1, 256=data-256 number = 256 , length = 1264 refCount = 1, 92=data-92 number = 92 , length = 1264 refCount = 1, 404=data-404 number = 404 , length = 1266 refCount = 1}
2008-06-12 14:55:58,563 [age Thread Pool] ERROR AbstractStoreCursor            - Failed to fill batch
java.io.IOException: Failed to read to journal for: offset = 0, file = 551, size = -1, type = 0. Reason: java.io.IOException: Could not locate data file data--551
	at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:33)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.createReadException(AMQPersistenceAdapter.java:635)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:521)
	at org.apache.activemq.store.amq.AMQMessageStore.getMessage(AMQMessageStore.java:432)
	at org.apache.activemq.store.amq.RecoveryListenerAdapter.recoverMessageReference(RecoveryListenerAdapter.java:54)
	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverReference(KahaReferenceStore.java:82)
	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverNextMessages(KahaReferenceStore.java:120)
	at org.apache.activemq.store.amq.AMQMessageStore.recoverNextMessages(AMQMessageStore.java:530)
	at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:75)
	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:188)
	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.onUsageChanged(AbstractStoreCursor.java:157)
	at org.apache.activemq.usage.Usage$1.run(Usage.java:266)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
	at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.IOException: Could not locate data file data--551
	at org.apache.activemq.kaha.impl.async.AsyncDataManager.getDataFile(AsyncDataManager.java:303)
	at org.apache.activemq.kaha.impl.async.AsyncDataManager.read(AsyncDataManager.java:613)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:518)
	... 12 more
2008-06-12 14:55:58,563 [age Thread Pool] ERROR AbstractStoreCursor            - Failed to fill batch 
java.lang.RuntimeException: java.io.IOException: Failed to read to journal for: offset = 0, file = 551, size = -1, type = 0. Reason: java.io.IOException: Could not locate data file data--551
	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:191)
	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.onUsageChanged(AbstractStoreCursor.java:157)
	at org.apache.activemq.usage.Usage$1.run(Usage.java:266)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
	at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.IOException: Failed to read to journal for: offset = 0, file = 551, size = -1, type = 0. Reason: java.io.IOException: Could not locate data file data--551
	at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:33)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.createReadException(AMQPersistenceAdapter.java:635)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:521)
	at org.apache.activemq.store.amq.AMQMessageStore.getMessage(AMQMessageStore.java:432)
	at org.apache.activemq.store.amq.RecoveryListenerAdapter.recoverMessageReference(RecoveryListenerAdapter.java:54)
	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverReference(KahaReferenceStore.java:82)
	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverNextMessages(KahaReferenceStore.java:120)
	at org.apache.activemq.store.amq.AMQMessageStore.recoverNextMessages(AMQMessageStore.java:530)
	at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:75)
	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:188)
	... 5 more
Caused by: java.io.IOException: Could not locate data file data--551
	at org.apache.activemq.kaha.impl.async.AsyncDataManager.getDataFile(AsyncDataManager.java:303)
	at org.apache.activemq.kaha.impl.async.AsyncDataManager.read(AsyncDataManager.java:613)
	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:518)
	... 12 more
2008-06-12 15:00:18,531 [main           ] INFO  BrokerService                  - ActiveMQ Message Broker (localhost, ID:salthill-2790-1213278917752-0:0) is shutting down
2008-06-12 15:00:20,438 [main           ] INFO  TransportConnector             - Connector Default Stopped
2008-06-12 15:00:20,641 [main           ] INFO  BrokerService                  - ActiveMQ JMS Message Broker (localhost, ID:salthill-2790-1213278917752-0:0) stopped



remove some unnecessary info from the description

> in use data files removed from data store under load
> ----------------------------------------------------
>
>                 Key: AMQ-1795
>                 URL: https://issues.apache.org/activemq/browse/AMQ-1795
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.1.0
>         Environment: all
>            Reporter: Gary Tully
>            Assignee: Rob Davies
>             Fix For: 5.2.0
>
>         Attachments: AMQ-1795.test_and_partial_fix
>
>
> Under load, I see the error below.
> the problem is that org.apache.activemq.store.amq.AMQPersistenceAdapter.cleanup() does not capture all of the in use files.
> I have made some changes that improves the situation but there is still a window, that seems to be related to spooling messages when a memory limit is reached.
> 2008-06-12 14:55:16,689 [main           ] INFO  BrokerService                  - Using Persistence Adapter: AMQPersistenceAdapter(activemq-data\localhost)
> 2008-06-12 14:55:16,704 [main           ] INFO  AMQPersistenceAdapter          - AMQStore starting using directory: activemq-data\localhost
> 2008-06-12 14:55:17,361 [main           ] INFO  AMQPersistenceAdapter          - Journal deleted: 
> 2008-06-12 14:55:17,361 [main           ] INFO  KahaStore                      - Kaha Store successfully deleted data directory activemq-data\localhost\kr-store\data
> 2008-06-12 14:55:17,392 [main           ] INFO  KahaStore                      - Kaha Store successfully deleted data directory activemq-data\localhost\kr-store\state
> 2008-06-12 14:55:17,392 [main           ] INFO  KahaStore                      - Kaha Store using data directory activemq-data\localhost\kr-store\state
> 2008-06-12 14:55:17,470 [main           ] INFO  AMQPersistenceAdapter          - Active data files: []
> 2008-06-12 14:55:17,705 [main           ] INFO  BrokerService                  - ActiveMQ null JMS Message Broker (localhost) is starting
> 2008-06-12 14:55:17,705 [main           ] INFO  BrokerService                  - For help or more information please see: http://activemq.apache.org/
> 2008-06-12 14:55:17,814 [JMX connector  ] INFO  ManagementContext              - JMX consoles can connect to service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
> 2008-06-12 14:55:17,924 [main           ] INFO  KahaStore                      - Kaha Store using data directory activemq-data\localhost\kr-store\data
> 2008-06-12 14:55:17,939 [main           ] INFO  TransportServerThreadSupport   - Listening for connections at: tcp://salthill:61616
> 2008-06-12 14:55:17,939 [main           ] INFO  TransportConnector             - Connector Default Started
> 2008-06-12 14:55:17,939 [main           ] INFO  BrokerService                  - ActiveMQ JMS Message Broker (localhost, ID:salthill-2790-1213278917752-0:0) started
> 2008-06-12 14:55:17,939 [main           ] INFO  MissingDataFileTest            - Starting broker..
> 2008-06-12 14:55:58,516 [age Thread Pool] ERROR AsyncDataManager               - Looking for key 551 but not found in fileMap: {1014=data-1014 number = 1014 , length = 1960 refCount = 6, 607=data-607 number = 607 , length = 1266 refCount = 1, 928=data-928 number = 928 , .. .. 622 , length = 1352 refCount = 2, 328=data-328 number = 328 , length = 1266 refCount = 1, 256=data-256 number = 256 , length = 1264 refCount = 1, 92=data-92 number = 92 , length = 1264 refCount = 1, 404=data-404 number = 404 , length = 1266 refCount = 1}
> 2008-06-12 14:55:58,563 [age Thread Pool] ERROR AbstractStoreCursor            - Failed to fill batch
> java.io.IOException: Failed to read to journal for: offset = 0, file = 551, size = -1, type = 0. Reason: java.io.IOException: Could not locate data file data--551
> 	at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:33)
> 	at org.apache.activemq.store.amq.AMQPersistenceAdapter.createReadException(AMQPersistenceAdapter.java:635)
> 	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:521)
> 	at org.apache.activemq.store.amq.AMQMessageStore.getMessage(AMQMessageStore.java:432)
> 	at org.apache.activemq.store.amq.RecoveryListenerAdapter.recoverMessageReference(RecoveryListenerAdapter.java:54)
> 	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverReference(KahaReferenceStore.java:82)
> 	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverNextMessages(KahaReferenceStore.java:120)
> 	at org.apache.activemq.store.amq.AMQMessageStore.recoverNextMessages(AMQMessageStore.java:530)
> 	at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:75)
> 	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:188)
> 	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.onUsageChanged(AbstractStoreCursor.java:157)
> 	at org.apache.activemq.usage.Usage$1.run(Usage.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> 	at java.lang.Thread.run(Thread.java:595)
> Caused by: java.io.IOException: Could not locate data file data--551
> 	at org.apache.activemq.kaha.impl.async.AsyncDataManager.getDataFile(AsyncDataManager.java:303)
> 	at org.apache.activemq.kaha.impl.async.AsyncDataManager.read(AsyncDataManager.java:613)
> 	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:518)
> 	... 12 more
> 2008-06-12 14:55:58,563 [age Thread Pool] ERROR AbstractStoreCursor            - Failed to fill batch 
> java.lang.RuntimeException: java.io.IOException: Failed to read to journal for: offset = 0, file = 551, size = -1, type = 0. Reason: java.io.IOException: Could not locate data file data--551
> 	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:191)
> 	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.onUsageChanged(AbstractStoreCursor.java:157)
> 	at org.apache.activemq.usage.Usage$1.run(Usage.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> 	at java.lang.Thread.run(Thread.java:595)
> Caused by: java.io.IOException: Failed to read to journal for: offset = 0, file = 551, size = -1, type = 0. Reason: java.io.IOException: Could not locate data file data--551
> 	at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:33)
> 	at org.apache.activemq.store.amq.AMQPersistenceAdapter.createReadException(AMQPersistenceAdapter.java:635)
> 	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:521)
> 	at org.apache.activemq.store.amq.AMQMessageStore.getMessage(AMQMessageStore.java:432)
> 	at org.apache.activemq.store.amq.RecoveryListenerAdapter.recoverMessageReference(RecoveryListenerAdapter.java:54)
> 	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverReference(KahaReferenceStore.java:82)
> 	at org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverNextMessages(KahaReferenceStore.java:120)
> 	at org.apache.activemq.store.amq.AMQMessageStore.recoverNextMessages(AMQMessageStore.java:530)
> 	at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:75)
> 	at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:188)
> 	... 5 more
> Caused by: java.io.IOException: Could not locate data file data--551
> 	at org.apache.activemq.kaha.impl.async.AsyncDataManager.getDataFile(AsyncDataManager.java:303)
> 	at org.apache.activemq.kaha.impl.async.AsyncDataManager.read(AsyncDataManager.java:613)
> 	at org.apache.activemq.store.amq.AMQPersistenceAdapter.readCommand(AMQPersistenceAdapter.java:518)
> 	... 12 more
> 2008-06-12 15:00:18,531 [main           ] INFO  BrokerService                  - ActiveMQ Message Broker (localhost, ID:salthill-2790-1213278917752-0:0) is shutting down
> 2008-06-12 15:00:20,438 [main           ] INFO  TransportConnector             - Connector Default Stopped
> 2008-06-12 15:00:20,641 [main           ] INFO  BrokerService                  - ActiveMQ JMS Message Broker (localhost, ID:salthill-2790-1213278917752-0:0) stopped

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message