asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Taewoo Kim (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ASTERIXDB-1900) Transaction log files are not truncated.
Date Tue, 02 May 2017 19:11:04 GMT

     [ https://issues.apache.org/jira/browse/ASTERIXDB-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Taewoo Kim updated ASTERIXDB-1900:
----------------------------------
    Description: 
On the current master, transaction log files are not truncated when I ingest more than millions
of records. I have a 10-nodes cluster (each has 4 partitions). Before restarting, the transaction
log files are never deleted on one node. 

For example, the following shows that the transaction files are never deleted since a feed
ingestion has started on May 1, 17:37. This happens on only one node. 

{code}
-rw-r--r-- 1 taewok2 grad  130 May  2 12:03 checkpoint_1493751802512
-rw-r--r-- 1 taewok2 grad 1.0G May  1 21:23 transaction_log_10
-rw-r--r-- 1 taewok2 grad 1.0G May  1 22:09 transaction_log_11
-rw-r--r-- 1 taewok2 grad 1.0G May  1 22:57 transaction_log_12
-rw-r--r-- 1 taewok2 grad 1.0G May  1 23:48 transaction_log_13
-rw-r--r-- 1 taewok2 grad 1.0G May  2 00:37 transaction_log_14
-rw-r--r-- 1 taewok2 grad 1.0G May  2 01:21 transaction_log_15
-rw-r--r-- 1 taewok2 grad 1.0G May  2 02:07 transaction_log_16
-rw-r--r-- 1 taewok2 grad 1.0G May  2 02:52 transaction_log_17
-rw-r--r-- 1 taewok2 grad 1.0G May  2 03:41 transaction_log_18
-rw-r--r-- 1 taewok2 grad 1.0G May  2 04:27 transaction_log_19
-rw-r--r-- 1 taewok2 grad 1.0G May  2 05:11 transaction_log_20
-rw-r--r-- 1 taewok2 grad 1.0G May  2 06:00 transaction_log_21
-rw-r--r-- 1 taewok2 grad 1.0G May  2 06:45 transaction_log_22
-rw-r--r-- 1 taewok2 grad 1.0G May  2 07:33 transaction_log_23
-rw-r--r-- 1 taewok2 grad 1.0G May  2 08:18 transaction_log_24
-rw-r--r-- 1 taewok2 grad 1.0G May  2 09:08 transaction_log_25
-rw-r--r-- 1 taewok2 grad 1.0G May  2 09:54 transaction_log_26
-rw-r--r-- 1 taewok2 grad 1.0G May  2 10:44 transaction_log_27
-rw-r--r-- 1 taewok2 grad 1.0G May  2 11:32 transaction_log_28
-rw-r--r-- 1 taewok2 grad 654M May  2 12:04 transaction_log_29
-rw-r--r-- 1 taewok2 grad 1.0G May  1 17:37 transaction_log_3
-rw-r--r-- 1 taewok2 grad 1.0G May  1 18:03 transaction_log_4
-rw-r--r-- 1 taewok2 grad 1.0G May  1 18:33 transaction_log_5
-rw-r--r-- 1 taewok2 grad 1.0G May  1 19:03 transaction_log_6
-rw-r--r-- 1 taewok2 grad 1.0G May  1 19:29 transaction_log_7
-rw-r--r-- 1 taewok2 grad 1.0G May  1 19:56 transaction_log_8
-rw-r--r-- 1 taewok2 grad 1.0G May  1 20:39 transaction_log_9
{code}

  was:
On the current master, transaction log files are not truncated when I ingest more than millions
of records. I have 10-nodes cluster (each has 4 partitions on SSDs). Before restarting, the
transaction log files are never deleted. 

For example, the following shows that the transaction files are never deleted since a feed
ingestion has started on May 1, 17:37. This happens on only one node. 

{code}
-rw-r--r-- 1 taewok2 grad  130 May  2 12:03 checkpoint_1493751802512
-rw-r--r-- 1 taewok2 grad 1.0G May  1 21:23 transaction_log_10
-rw-r--r-- 1 taewok2 grad 1.0G May  1 22:09 transaction_log_11
-rw-r--r-- 1 taewok2 grad 1.0G May  1 22:57 transaction_log_12
-rw-r--r-- 1 taewok2 grad 1.0G May  1 23:48 transaction_log_13
-rw-r--r-- 1 taewok2 grad 1.0G May  2 00:37 transaction_log_14
-rw-r--r-- 1 taewok2 grad 1.0G May  2 01:21 transaction_log_15
-rw-r--r-- 1 taewok2 grad 1.0G May  2 02:07 transaction_log_16
-rw-r--r-- 1 taewok2 grad 1.0G May  2 02:52 transaction_log_17
-rw-r--r-- 1 taewok2 grad 1.0G May  2 03:41 transaction_log_18
-rw-r--r-- 1 taewok2 grad 1.0G May  2 04:27 transaction_log_19
-rw-r--r-- 1 taewok2 grad 1.0G May  2 05:11 transaction_log_20
-rw-r--r-- 1 taewok2 grad 1.0G May  2 06:00 transaction_log_21
-rw-r--r-- 1 taewok2 grad 1.0G May  2 06:45 transaction_log_22
-rw-r--r-- 1 taewok2 grad 1.0G May  2 07:33 transaction_log_23
-rw-r--r-- 1 taewok2 grad 1.0G May  2 08:18 transaction_log_24
-rw-r--r-- 1 taewok2 grad 1.0G May  2 09:08 transaction_log_25
-rw-r--r-- 1 taewok2 grad 1.0G May  2 09:54 transaction_log_26
-rw-r--r-- 1 taewok2 grad 1.0G May  2 10:44 transaction_log_27
-rw-r--r-- 1 taewok2 grad 1.0G May  2 11:32 transaction_log_28
-rw-r--r-- 1 taewok2 grad 654M May  2 12:04 transaction_log_29
-rw-r--r-- 1 taewok2 grad 1.0G May  1 17:37 transaction_log_3
-rw-r--r-- 1 taewok2 grad 1.0G May  1 18:03 transaction_log_4
-rw-r--r-- 1 taewok2 grad 1.0G May  1 18:33 transaction_log_5
-rw-r--r-- 1 taewok2 grad 1.0G May  1 19:03 transaction_log_6
-rw-r--r-- 1 taewok2 grad 1.0G May  1 19:29 transaction_log_7
-rw-r--r-- 1 taewok2 grad 1.0G May  1 19:56 transaction_log_8
-rw-r--r-- 1 taewok2 grad 1.0G May  1 20:39 transaction_log_9
{code}


> Transaction log files are not truncated.
> ----------------------------------------
>
>                 Key: ASTERIXDB-1900
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1900
>             Project: Apache AsterixDB
>          Issue Type: Bug
>            Reporter: Taewoo Kim
>
> On the current master, transaction log files are not truncated when I ingest more than
millions of records. I have a 10-nodes cluster (each has 4 partitions). Before restarting,
the transaction log files are never deleted on one node. 
> For example, the following shows that the transaction files are never deleted since a
feed ingestion has started on May 1, 17:37. This happens on only one node. 
> {code}
> -rw-r--r-- 1 taewok2 grad  130 May  2 12:03 checkpoint_1493751802512
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 21:23 transaction_log_10
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 22:09 transaction_log_11
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 22:57 transaction_log_12
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 23:48 transaction_log_13
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 00:37 transaction_log_14
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 01:21 transaction_log_15
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 02:07 transaction_log_16
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 02:52 transaction_log_17
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 03:41 transaction_log_18
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 04:27 transaction_log_19
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 05:11 transaction_log_20
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 06:00 transaction_log_21
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 06:45 transaction_log_22
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 07:33 transaction_log_23
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 08:18 transaction_log_24
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 09:08 transaction_log_25
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 09:54 transaction_log_26
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 10:44 transaction_log_27
> -rw-r--r-- 1 taewok2 grad 1.0G May  2 11:32 transaction_log_28
> -rw-r--r-- 1 taewok2 grad 654M May  2 12:04 transaction_log_29
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 17:37 transaction_log_3
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 18:03 transaction_log_4
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 18:33 transaction_log_5
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 19:03 transaction_log_6
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 19:29 transaction_log_7
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 19:56 transaction_log_8
> -rw-r--r-- 1 taewok2 grad 1.0G May  1 20:39 transaction_log_9
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message