hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7213) Have HLog files for .META. edits only
Date Tue, 04 Dec 2012 06:16:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509537#comment-13509537
] 

stack commented on HBASE-7213:
------------------------------

.META. file ends in '.META' and not '.META.'?  Can't it end in .log?  xxxxxx.log.META looks
a bit odd and I'm sure will trip up someone else trying to find WALs for whatever reason.

How does splitMetaLog work?  It is passed a server name and if we find logs in the filesystem,
then we'll split its logs?  What if .META. is currently assigned? The splitting will be to
no avail?  When would we ever log this message:

{code}
+      LOG.info("No logs to split");
{code}

Should that be .META. logs to split?

Methods usually have a space between them rather than:

{code}
+  }
+  /**
{code}

... in hbase codebase boss.

Why emit two log lines when you could have one that varies with whether meta or not:

{code}
             LOG.info("Done splitting " + path);
+            if (path.endsWith(HConstants.META_HLOG_FILE_EXTN)) {
+              LOG.info("Done splitting meta region");
+            }
{code}

Should be a isMetaWAL method?  to do the looking for META_LOG_FILE_EXTN... since used in a
few places.

In process, we have code for meta and root handling in the general ServerShutdownHandler.
 Should the root handling be up in RootServerShutdownHandler, in its process, which then calls
through to the general SSH#process and ditto for the .META.?  (Might mean ROOT needs to subclass
Meta for now... ).  We maybe should have done it before but was a bit tougher since had to
do the log splitting first...  could have broke up the process more ... but might be easier
now these root and meta handlings are first thing done in the process method.

Is this needed?

{code}
-        this.services.getExecutorService().submit(this);
+        //typecast to SSH so that we don't do the above meta log split again
+        this.services.getExecutorService().submit((ServerShutdownHandler)this);
{code}

Would make sense if you move the meta specific code to metaSSH, and ditto on ROOT?

MetaServices Interface doesn't seem right?  Its all about a WAL.  What else would ever go
in here?  Who uses this new Interface and its methods?  I don't see anything else in this
patch using it.  I suppose MetaLogRoller  does but it is sitting beside RegionServer -- could
be protected or package private -- yet its public in the Interface.  Hmm... i see that OpenRegionHandler
uses it.  Can't RegionServer notice when its a meta region and do the setup etc., internally?
 Ugh, maybe not looking at this OpenRegionHandler....how it needs the exact WAL?  What if
you passed in the regionname to getWAL?  Then it internally would do the right thing.  You
wouldn't have to add new methods?   Especially public ones?

When we lose the .META. region, does the RS clean up its .META. log?

I don't like extending the RegionServerServices because makes it harder mocking up an RSS...
more stuff to add in.

Should it be MetaFSHLog rather than FSLog w/ boolean args in constructor?  Just wondering.

You have this: +        LOG.info("CREATED writer " + path); //REMOVETHIS

We have to add this method?

+    public static HLog createMetaHLog(final FileSystem fs, final Path root, final String
logName,
+        final Configuration conf, final List<WALActionsListener> listeners,
+        final String prefix) throws IOException {
+      return new FSHLog(fs, root, logName, HConstants.HREGION_OLDLOGDIR_NAME, 
+            conf, listeners, false, prefix, true);
+    }

Nothing in say the log name that would clue us its a .META. file?  Could we preface the logName
with .META. or something and then we don't have to pass flags to FSHLog?  Could have static
isMetaWAL method that looks at log name and can tell it .META.?

Import this in LogSplitter but not used: +import org.apache.hadoop.fs.PathFilter;

Is this used:

-  private static final Pattern pattern = Pattern.compile(".*\\.\\d*");
+  private static final Pattern pattern = Pattern.compile(".*\\.\\d*(.META)*");

Good stuff.
                
> Have HLog files for .META. edits only
> -------------------------------------
>
>                 Key: HBASE-7213
>                 URL: https://issues.apache.org/jira/browse/HBASE-7213
>             Project: HBase
>          Issue Type: Improvement
>          Components: master, regionserver
>            Reporter: Devaraj Das
>            Assignee: Devaraj Das
>             Fix For: 0.96.0
>
>         Attachments: 7213-in-progress.2.2.patch, 7213-in-progress.2.patch, 7213-in-progress.patch
>
>
> Over on HBASE-6774, there is a discussion on separating out the edits for .META. regions
from the other regions' edits w.r.t where the edits are written. This jira is to track an
implementation of that.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message