Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <744025753.1261303458064.JavaMail.jira@brutus>
Date: Sun, 20 Dec 2009 10:04:18 +0000 (UTC)
From: "Lars George (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-2055) Serialize WAL as Avro records
In-Reply-To: <404475344.1261160538266.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-2055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792979#action_12792979 ] 

Lars George commented on HBASE-2055:
------------------------------------

Great stuff Andy, this block marker makes totally sense also in the context of log splitting, which needs blocks of the log being presorted by the RS's before they get applied. With the marker this is a natural fit. BigTable does 64k chunks too (during sorts).

> Serialize WAL as Avro records
> -----------------------------
>
>                 Key: HBASE-2055
>                 URL: https://issues.apache.org/jira/browse/HBASE-2055
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>            Priority: Minor
>         Attachments: HBASE-2055-v2.patch, HBASE-2055-v3.patch, HBASE-2055.patch, jackson-core-asl-1.0.1.jar, jackson-mapper-asl-1.0.1.jar, paranamer-1.5.jar, TEST-org.apache.hadoop.hbase.regionserver.wal.TestHLog.txt.gz, TEST-org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.txt.gz, TEST-org.apache.hadoop.hbase.TestFullLogReconstruction.txt.gz, test-site.patch
>
>
> There was some advocacy of using Avro for serialization of HBase WAL records up on hbase-dev@. Idea is Hadoop core is getting away from Writables and Avro is the blessed replacement. 
> I think we have this criteria for its use:
> 1) Performance of writing Avro records is no worse than that for writing Writables into a SequenceFile.
> 2) Space consumed by Avro serialization is no worse than that of Writables
> 3) File format is amenable to appends (cannot require valid trailers, etc.)
> I'll put up a patch so we can try it out. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.