hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Hammerbacher (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2055) Serialize WAL as Avro records
Date Fri, 25 Dec 2009 22:24:29 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794573#action_12794573

Jeff Hammerbacher commented on HBASE-2055:

Hey Andy,

I don't think putting a snapshot of trunk into your svn would be a great idea. The 1.3 release
will have a finalized design and implementation of the file object container.

As for the configurable SYNC_INTERVAL: it makes sense to me to make this a configuration parameter.
That said, it would be worth raising your concerns on the AVRO JIRA rather than in this issue.


> Serialize WAL as Avro records
> -----------------------------
>                 Key: HBASE-2055
>                 URL: https://issues.apache.org/jira/browse/HBASE-2055
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-2055-v2.patch, HBASE-2055-v3.patch, HBASE-2055-v4.patch, HBASE-2055.patch,
jackson-core-asl-1.0.1.jar, jackson-mapper-asl-1.0.1.jar, paranamer-1.5.jar, TEST-org.apache.hadoop.hbase.regionserver.wal.TestHLog.txt.gz,
TEST-org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.txt.gz, TEST-org.apache.hadoop.hbase.TestFullLogReconstruction.txt.gz,
> There was some advocacy of using Avro for serialization of HBase WAL records up on hbase-dev@.
Idea is Hadoop core is getting away from Writables and Avro is the blessed replacement. 
> I think we have this criteria for its use:
> 1) Performance of writing Avro records is no worse than that for writing Writables into
a SequenceFile.
> 2) Space consumed by Avro serialization is no worse than that of Writables
> 3) File format is amenable to appends (cannot require valid trailers, etc.)
> I'll put up a patch so we can try it out. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message