hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16094) Procedure v2 - Improve cleaning up of proc wals
Date Thu, 02 Nov 2017 20:35:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16236545#comment-16236545
] 

Ted Yu commented on HBASE-16094:
--------------------------------

This seems applicable to branch-1 as well.

Mind if I open a backport JIRA ?

> Procedure v2 - Improve cleaning up of proc wals
> -----------------------------------------------
>
>                 Key: HBASE-16094
>                 URL: https://issues.apache.org/jira/browse/HBASE-16094
>             Project: HBase
>          Issue Type: Sub-task
>          Components: proc-v2
>            Reporter: Appy
>            Assignee: Appy
>            Priority: Major
>             Fix For: 2.0.0
>
>         Attachments: HBASE-16094.master.001.patch, HBASE-16094.master.002.patch, HBASE-16094.master.003.patch,
HBASE-16094.master.004.patch, HBASE-16094.master.005.patch, HBASE-16094.master.006.patch
>
>
> Avoid accumulating too many wals.
> We remove logs in 3 cases:
>  - No procedures are running. We can remove all the logs.
>  - All procedures are updated/written in the last log. We can remove all the logs, aside
the active one.	
>  - Remove log if does not contains “active” procIds
> https://github.com/apache/hbase/blob/master/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java#L865
> The last point, Remove log if does not contains “active” procIds, needs to be improved.
Basically, even if a log contains state of an active proc, we can delete it if a later log
contains a newer state.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message