asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ian Maxon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ASTERIXDB-1900) Transaction log files are not truncated.
Date Wed, 03 May 2017 03:34:04 GMT

    [ https://issues.apache.org/jira/browse/ASTERIXDB-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994228#comment-15994228
] 

Ian Maxon commented on ASTERIXDB-1900:
--------------------------------------

This is related to an issue with incorrect resource use counting on the Metadata indexes.
The methods in MetadataNode enter the PrimaryIndexOperationTracker but only the index that
is part of the MetadataTransaction has its reference count decremented through the normal
callback via LogBuffer.  I have a fix but I think the implications require some thought as
to how it affects the consistency design of the Metadata. A work around is to simply restart
after modifying anything in Metadata, this way the reference counts are at 0 and soft checkpoints
will proceed as intended. 

> Transaction log files are not truncated.
> ----------------------------------------
>
>                 Key: ASTERIXDB-1900
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1900
>             Project: Apache AsterixDB
>          Issue Type: Bug
>            Reporter: Taewoo Kim
>            Assignee: Ian Maxon
>
> On the current master, transaction log files are not truncated when I ingest more than
millions of records. I have a 10-nodes cluster (each has 4 partitions). Before restarting,
the transaction log files are never deleted on one node. 
> For example, the following shows that the transaction files are never deleted since a
feed ingestion has started on May 1, 17:37. This happens on only one node. 
> {code}
> -rw-r--r-- 1 taewok2 grad  130 May  2 12:03 checkpoint_1493751802512
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 21:23 transaction_log_10
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 22:09 transaction_log_11
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 22:57 transaction_log_12
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 23:48 transaction_log_13
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 00:37 transaction_log_14
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 01:21 transaction_log_15
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 02:07 transaction_log_16
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 02:52 transaction_log_17
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 03:41 transaction_log_18
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 04:27 transaction_log_19
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 05:11 transaction_log_20
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 06:00 transaction_log_21
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 06:45 transaction_log_22
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 07:33 transaction_log_23
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 08:18 transaction_log_24
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 09:08 transaction_log_25
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 09:54 transaction_log_26
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 10:44 transaction_log_27
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 11:32 transaction_log_28
> -rw-r--r-- 1 taewok2 grad 654M May  2 12:04 transaction_log_29
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 17:37 transaction_log_3
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 18:03 transaction_log_4
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 18:33 transaction_log_5
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 19:03 transaction_log_6
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 19:29 transaction_log_7
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 19:56 transaction_log_8
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 20:39 transaction_log_9
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message