hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Hammerbacher (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2055) Serialize WAL as Avro records
Date Wed, 23 Dec 2009 19:38:29 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794181#action_12794181
] 

Jeff Hammerbacher commented on HBASE-2055:
------------------------------------------

Hey Andy,

Grab the latest Java patch from https://issues.apache.org/jira/browse/AVRO-160--the new file
format puts metadata in the header, rather than the footer. For future maintenance, it may
be easier to stick with the default Avro file object container.

Later,
Jeff

> Serialize WAL as Avro records
> -----------------------------
>
>                 Key: HBASE-2055
>                 URL: https://issues.apache.org/jira/browse/HBASE-2055
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-2055-v2.patch, HBASE-2055-v3.patch, HBASE-2055-v4.patch, HBASE-2055.patch,
jackson-core-asl-1.0.1.jar, jackson-mapper-asl-1.0.1.jar, paranamer-1.5.jar, TEST-org.apache.hadoop.hbase.regionserver.wal.TestHLog.txt.gz,
TEST-org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.txt.gz, TEST-org.apache.hadoop.hbase.TestFullLogReconstruction.txt.gz,
test-site.patch
>
>
> There was some advocacy of using Avro for serialization of HBase WAL records up on hbase-dev@.
Idea is Hadoop core is getting away from Writables and Avro is the blessed replacement. 
> I think we have this criteria for its use:
> 1) Performance of writing Avro records is no worse than that for writing Writables into
a SequenceFile.
> 2) Space consumed by Avro serialization is no worse than that of Writables
> 3) File format is amenable to appends (cannot require valid trailers, etc.)
> I'll put up a patch so we can try it out. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message