hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars George (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1994) Master will lose hlog entries while splitting if region has empty oldlogfile.log
Date Thu, 19 Nov 2009 15:29:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780025#action_12780025

Lars George commented on HBASE-1994:

This could simply follow or use the normal file name conventions for hlogs:
        this.filenum = System.currentTimeMillis();
        Path newPath = computeFilename(this.filenum);

  public Path computeFilename(final long fn) {
    if (fn < 0) return null;
    return new Path(dir, HLOG_DATFILE + fn);

Just with the known extension. Then in RS and Store classes change

  /** Name of old log file for reconstruction */
  static final String HREGION_OLDLOGFILE_NAME = "oldlogfile.log";

to be a extension only and not pass this on along the Store initialization but scan the FS
for such file at the right spot in the code. Seems cleaner too, as it removes at least one

> Master will lose hlog entries while splitting if region has empty oldlogfile.log
> --------------------------------------------------------------------------------
>                 Key: HBASE-1994
>                 URL: https://issues.apache.org/jira/browse/HBASE-1994
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.21.0
>            Reporter: Cosmin Lehene
>            Priority: Blocker
>             Fix For: 0.21.0
>   Original Estimate: 48h
>  Remaining Estimate: 48h
> I don't know yet how an empty oldlogfile.log can exist, however it happened.
> Master will fail to put the splits in the region oldlogfile.log if an empty oldlogfile.log
already exists there.
> This is the master log after I artificially reproduced it by placing an empty oldlogfile.log
in /hbase/.META./1028785192/oldlogfile.log and then killed the regionserver that was holding
the .META. table
> 2009-11-19 09:08:36,012 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Splitting
1 hlog(s) in hdfs://b0:9000/hbase/.logs/b4,60020,1258637492773
> 2009-11-19 09:08:36,012 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: Splitting
hlog 1 of 1: hdfs://b0:9000/hbase/.logs/b4,60020,1258637492773/hlog.dat.1258637493128, length=0
> 2009-11-19 09:08:36,019 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: Adding queue
for .META.,,1
> 2009-11-19 09:08:36,037 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: Pushed=795
entries from hdfs://b0:9000/hbase/.logs/b4,60020,1258637492773/hlog.dat.1258637493128
> 2009-11-19 09:08:36,038 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog: Thread got
795 to process
> 2009-11-19 09:08:36,043 WARN org.apache.hadoop.hbase.regionserver.wal.HLog: Old hlog
file hdfs://b0:9000/hbase/.META./1028785192/oldlogfile.log already exists. Copying existing
file to new file
> 2009-11-19 09:08:36,079 WARN org.apache.hadoop.hbase.regionserver.wal.HLog: Got while
writing region .META.,,1 log java.io.EOFException
> 2009-11-19 09:08:36,081 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: hlog file
splitting completed in 70 millis for hdfs://b0:9000/hbase/.logs/b4,60020,1258637492773

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message