activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eduardo Costa (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AMQ-2736) KahaDB doesn't clean up old files
Date Wed, 13 Oct 2010 17:22:41 GMT

    [ https://issues.apache.org/activemq/browse/AMQ-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=62560#action_62560
] 

Eduardo Costa commented on AMQ-2736:
------------------------------------

After some evil debugging tricks, I found that there are some pending inflight transactions
(even after restarting the server) in my database files. In this scenario, the gcCandidadeSet
is empty, since "metadata.firstInProgressTransactionLocation" points to the first log file
(MessageDatabase.java, lines 1140-1154):

{code}// Don't GC files after the first in progress tx
Location firstTxLocation = metadata.lastUpdate;
if( metadata.firstInProgressTransactionLocation!=null ) {
  firstTxLocation = metadata.firstInProgressTransactionLocation;
}

if( firstTxLocation!=null ) {
  while( !gcCandidateSet.isEmpty() ) {
    Integer last = gcCandidateSet.last();
    if( last >= firstTxLocation.getDataFileId() ) {
      gcCandidateSet.remove(last);
    } else {
      break;
    }
  }
}
{code}

This leads to the question: why it must discard every log file starting from the one containing
the first pending transaction? Any malfunctioning operation (even an one-time little mistake)
will trash all log files from that point! 

I believe this should happens also on "kill -9" or unexpected power down.

> KahaDB doesn't clean up old files
> ---------------------------------
>
>                 Key: AMQ-2736
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2736
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.3.2
>            Reporter: Adrian Trenaman
>            Priority: Critical
>         Attachments: AMQ-2736.zip, amq-2987-testcase.patch, amq-2987.patch, AMQ2736Test_should_with_this_diff.txt
>
>
> Over time, we're seeing that kahadb doesn't clean up old journal files. As a result,
we eventually run out of disk space, or rather, we hit our usage limits and our producers
are slowed down by the producer flow control mechanism. Others have experienced this problem
too (for example, see http://mail-archives.apache.org/mod_mbox/activemq-users/201002.mbox/%3C27502591.post@talk.nabble.com%3E)
> For now, we're moving back to the old amqPersistenceStore. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message