avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Percy <mpe...@apache.org>
Subject Avro container format WAL with the Java API
Date Tue, 09 Oct 2012 02:33:28 GMT
Hi all,
I am trying to implement a very simple WAL using the Avro container format.
Since my records are small, at startup time I seek backward by 256 bytes
from the end of the file before calling DataFileReader.sync(), and then run
through the available records to find the last one. I am writing a sync
marker and flushing after every record in with my writer. I was just
wondering, does this seem like a reasonable approach? Or is there a better
way to simply read the last record from the file?

Code snippet:

  private final DataFileReader<TransferStateFileMeta> reader;
  // ...

  /** Read the last record in the file */
  private void initReader() throws IOException {
    long syncPos = metaFile.length() - 256L;
    if (syncPos < 0) syncPos = 0L;
    reader.sync(syncPos);
    while (reader.hasNext()) {
      reader.next(metaCache);
    }
  }

Regards,
Mike

Mime
View raw message