hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Appy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-16094) Improve cleaning up of proc wals
Date Thu, 23 Jun 2016 20:17:16 GMT
Appy created HBASE-16094:
----------------------------

             Summary: Improve cleaning up of proc wals
                 Key: HBASE-16094
                 URL: https://issues.apache.org/jira/browse/HBASE-16094
             Project: HBase
          Issue Type: Bug
            Reporter: Appy


(From Matteo)
Avoid accumulating too many wals.
We remove logs in 3 cases:
No procedures are running. We can remove all the logs.
All procedures are updated/written in the last log. We can remove all the logs, aside the
active one.	
Remove log if does not contains “active” procIds (Medium)
https://github.com/apache/hbase/blob/master/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java#L865
This one requires to be improved, since those procedures can be in earlier wals. An example
that can be unit-tested is:
Insert 10 proc, Roll The Wal, Update the 10 proc
At the end of the 10 updates the removeInactiveLogs() should be able to remove the first WAL.
We can probably do a diff on the updated tracker bitmap between wals.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message