hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: .oldlogs in region dir
Date Wed, 06 Jul 2011 20:46:10 GMT
It looks like left-over crud Lars.  Thanks for noting it.  I made
HBASE-4068 to clean it up.
St.Ack

On Tue, Jul 5, 2011 at 7:41 AM, Lars George <lars.george@gmail.com> wrote:
> Hi Dhruba,
>
> Yes, but that is the one on the root dir level. I am referring to the one on
> the region level.
>
> Cheers,
> Lars
>
> On Tue, Jul 5, 2011 at 6:25 AM, Dhruba Borthakur <dhruba@gmail.com> wrote:
>
>> Doesn't the wide-area HBase replication use the .oldlogs to keep the slave
>> hBase cluster in sync?
>>
>> thanks
>> dhruba
>>
>> On Tue, Jul 5, 2011 at 7:27 AM, Lars George <lars.george@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > Ah, I see Ted has that also questioned in HBASE-4010. Good.
>> >
>> > And I was slightly wrong below, as there is a .logs and .oldlogs for the
>> > newly created regions. I agree that when there is no chance for the
>> region
>> > to take edits before it gets the proper log, then we should drop this.
>> Those
>> > dangling dirs do not harm, but also do not help either.
>> >
>> > Lars
>> >
>> > On Jul 5, 2011, at 8:39 AM, Lars George wrote:
>> >
>> > > Hi,
>> > >
>> > > Could someone confirm that we dropped the replay of the .oldlogs in the
>> > region directory because it is not needed anymore? Before the master
>> rewrite
>> > we were, just as now, creating the region and for a brief period it had
>> its
>> > own log directory, the .oldlogs under the region directory. The
>> > HRegion.initialize() would replay it in case an edit to the region
>> arrived
>> > in between. It would then also remove it:
>> > >
>> > >   public void initialize(Path initialFiles, final Progressable
>> reporter)
>> > >   throws IOException {
>> > >     Path oldLogFile = new Path(regiondir, HREGION_OLDLOGFILE_NAME);
>> > > ...
>> > >     // Play log if one.  Delete when done.
>> > >     doReconstructionLog(oldLogFile, minSeqIdToRecover, maxSeqId,
>> > reporter);
>> > >     if (fs.exists(oldLogFile)) {
>> > >       if (LOG.isDebugEnabled()) {
>> > >         LOG.debug("Deleting old log file: " + oldLogFile);
>> > >       }
>> > >       fs.delete(oldLogFile, false);
>> > >     }
>> > > ...
>> > >
>> > > Now we still create that directory, but the above code has been
>> dropped.
>> > Only the recovered.edits is replayed instead while the .oldlogs now hang
>> > around. I am not even sure when they may contain an entry, but if they
>> don't
>> > then we can either do not use "<regiondir>/.oldlogs", but point it to
the
>> > new "<regiondir>/.tmp" - or we can add a unconditional delete in the
>> > initialize as before.
>> > >
>> > > Create a JIRA for this?
>> > >
>> > > Lars
>> >
>> >
>>
>>
>> --
>> Connect to me at http://www.facebook.com/dhruba
>>
>

Mime
View raw message