hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-1897) WAL is spread all over code-base and presumption is its SequenceFile; encapsulate inside a wal subpackage
Date Fri, 09 Oct 2009 04:48:31 GMT
WAL is spread all over code-base and presumption is its SequenceFile; encapsulate inside a
wal subpackage
---------------------------------------------------------------------------------------------------------

                 Key: HBASE-1897
                 URL: https://issues.apache.org/jira/browse/HBASE-1897
             Project: Hadoop HBase
          Issue Type: Improvement
            Reporter: stack


Presumption is that WAL is a sequencefile.  I just spent some time looking again at my old
buddy SF and its kinda heavy-duty for our needs.  Do we need the sync bytes it writes into
the stream every time you call a sync?  Maybe it'd help recovering logs of edits?  We don't
need the compression by record or block, metadata info, etc.  Its also currently unsuited
because its sync actually doesn't do a sync on the backing stream:

{code}
    /** create a sync point */
    public void sync() throws IOException {
      if (sync != null && lastSyncPos != out.getPos()) {
        out.writeInt(SYNC_ESCAPE);                // mark the start of the sync
        out.write(sync);                          // write sync
        lastSyncPos = out.getPos();               // update lastSyncPos
      }
    }
{code}

We should move all of our HLog stuff into a wal package and rename the classes as WAL, WALEdit,
etc.  Splitting code and replay code should reference a WAL.Reader rather than a SequenceFile.Reader,
etc.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message