hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish Singhi <ashish.singhi.apa...@gmail.com>
Subject Re: Removing .oldlogs may lead to replication problem?
Date Fri, 07 Aug 2015 09:06:48 GMT
Hi.
> Could it be because of manually deletion of the WALs in .oldlogs?
Yes very much.
Ideally it is not suggested to remove the logs manually from .oldlogs. It
is auto cleaned by LogCleaner thread. If it is not cleaned up then there is
some reason for it.
There are cases when the WAL file is not replicated and it is moved to its
archive directory(.oldlogs). Then ReplicationLogCleaner thread will ensure
that this files are not cleaned up before it is replicated to another
cluster.
I hope I have answered your question.

Regards,
Ashish Singhi

On Fri, Aug 7, 2015 at 12:44 PM, Shuai Lin <linshuai2012@gmail.com> wrote:

> Hi all,
>
> We have two hbase cluster (one prod, one backup) running hbase 0.94.6 (from
> cdh4.) and have setup master-master replication.
>
> Last week we find the .oldlogs in the prod cluster was growing very large
> (13TB) and decided to remove it.
>
> But not until yesterday do we find that the replication had been in problem
> for almost a mongth due to some mis-configured firewall rules. After fixing
> the firewall the replication seems to be ok now, but some data which can be
> found in the prod cluster can't be found in the backup cluster. Could it be
> because of manually deletion of the WALs in .oldlogs?
>
> I have read a lot about replication and WALs, but could not be sure whether
> the logs in .oldlogs is related to replication.
>
> Can anyone share some thoughts? Thanks!
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message