activemq-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lionel Cons (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APLO-296) Improve LevelDB compaction
Date Thu, 21 Feb 2013 06:34:12 GMT

    [ https://issues.apache.org/jira/browse/APLO-296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582962#comment-13582962
] 

Lionel Cons commented on APLO-296:
----------------------------------

FWIW, on other brokers we do see long compaction times:

2013-02-20 08:09:28,387 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
2013-02-20 08:09:59,455 | INFO  | Compaction completed |

or

2013-02-20 08:08:18,655 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
2013-02-20 08:08:42,490 | INFO  | Compaction completed |

One more reason to be able to tune compaction...
                
> Improve LevelDB compaction
> --------------------------
>
>                 Key: APLO-296
>                 URL: https://issues.apache.org/jira/browse/APLO-296
>             Project: ActiveMQ Apollo
>          Issue Type: Improvement
>         Environment: apollo-99-trunk-20130208.033447-187
>            Reporter: Lionel Cons
>
> As a result of APLO-245, Apollo now automatically triggers LevelDB store compaction.
> However, on some brokers, we see this happening many times. See for instance:
> 2013-02-08 07:35:18,495 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:35:18,549 | INFO  | Compaction completed | 
> 2013-02-08 07:35:28,494 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:35:28,507 | INFO  | Compaction completed | 
> 2013-02-08 07:35:38,497 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:35:38,517 | INFO  | Compaction completed | 
> 2013-02-08 07:35:48,497 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:35:48,515 | INFO  | Compaction completed | 
> 2013-02-08 07:35:58,499 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:35:58,544 | INFO  | Compaction completed | 
> 2013-02-08 07:36:08,500 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:36:08,573 | INFO  | Compaction completed | 
> 2013-02-08 07:36:18,503 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:36:18,543 | INFO  | Compaction completed | 
> 2013-02-08 07:36:28,503 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:36:28,565 | INFO  | Compaction completed | 
> 2013-02-08 07:36:38,503 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:36:38,557 | INFO  | Compaction completed | 
> 2013-02-08 07:36:48,505 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:36:48,556 | INFO  | Compaction completed | 
> 2013-02-08 07:36:58,507 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:36:58,562 | INFO  | Compaction completed | 
> 2013-02-08 07:37:08,509 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:37:08,569 | INFO  | Compaction completed | 
> 2013-02-08 07:37:18,512 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:37:18,568 | INFO  | Compaction completed | 
> 2013-02-08 07:37:28,515 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:37:28,560 | INFO  | Compaction completed | 
> 2013-02-08 07:37:38,515 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:37:38,576 | INFO  | Compaction completed | 
> 2013-02-08 07:37:48,518 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:37:48,541 | INFO  | Compaction completed | 
> 2013-02-08 07:37:58,518 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:37:58,564 | INFO  | Compaction completed | 
> 2013-02-08 07:38:08,519 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:38:08,542 | INFO  | Compaction completed | 
> 2013-02-08 07:38:18,521 | INFO  | Compacting the leveldb index at: /var/lib/apollo/data/dirty.index
| 
> 2013-02-08 07:38:18,577 | INFO  | Compaction completed | 
> First, would it be possible to cut the number of log lines by 2 and log only one line
per compaction, with something like "Compacted the leveldb index at: /var/lib/apollo/data/dirty.index
in x.y ms"?
> Then, would it be possible to tune this auto-compaction so that it happens less frequently?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message