Return-Path: X-Original-To: apmail-pig-dev-archive@www.apache.org Delivered-To: apmail-pig-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B8F4DE71E for ; Sun, 17 Feb 2013 03:35:16 +0000 (UTC) Received: (qmail 85259 invoked by uid 500); 17 Feb 2013 03:35:16 -0000 Delivered-To: apmail-pig-dev-archive@pig.apache.org Received: (qmail 85192 invoked by uid 500); 17 Feb 2013 03:35:16 -0000 Mailing-List: contact dev-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list dev@pig.apache.org Received: (qmail 85180 invoked by uid 500); 17 Feb 2013 03:35:16 -0000 Delivered-To: apmail-hadoop-pig-dev@hadoop.apache.org Received: (qmail 85176 invoked by uid 99); 17 Feb 2013 03:35:16 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 17 Feb 2013 03:35:16 +0000 Date: Sun, 17 Feb 2013 03:35:16 +0000 (UTC) From: "Russell Jurney (JIRA)" To: pig-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (PIG-3015) Rewrite of AvroStorage MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PIG-3015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13580116#comment-13580116 ] Russell Jurney commented on PIG-3015: ------------------------------------- Which patch should I be testing? > Rewrite of AvroStorage > ---------------------- > > Key: PIG-3015 > URL: https://issues.apache.org/jira/browse/PIG-3015 > Project: Pig > Issue Type: Improvement > Components: piggybank > Reporter: Joseph Adler > Assignee: Joseph Adler > Attachments: bad.avro, good.avro, PIG-3015-2.patch, PIG-3015-3.patch, PIG-3015-4.patch, PIG-3015-5.patch, PIG-3015-6.patch, PIG-3015-7.patch, PIG-3015-8.patch, TestInput.java, Test.java > > > The current AvroStorage implementation has a lot of issues: it requires old versions of Avro, it copies data much more than needed, and it's verbose and complicated. (One pet peeve of mine is that old versions of Avro don't support Snappy compression.) > I rewrote AvroStorage from scratch to fix these issues. In early tests, the new implementation is significantly faster, and the code is a lot simpler. Rewriting AvroStorage also enabled me to implement support for Trevni (as TrevniStorage). > I'm opening this ticket to facilitate discussion while I figure out the best way to contribute the changes back to Apache. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira