Return-Path: Delivered-To: apmail-hive-user-archive@www.apache.org Received: (qmail 59877 invoked from network); 13 Nov 2010 22:30:31 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 13 Nov 2010 22:30:31 -0000 Received: (qmail 37472 invoked by uid 500); 13 Nov 2010 22:31:02 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 37400 invoked by uid 500); 13 Nov 2010 22:31:01 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 37392 invoked by uid 99); 13 Nov 2010 22:31:01 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 13 Nov 2010 22:31:01 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=10.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [128.105.6.39] (HELO sandstone.cs.wisc.edu) (128.105.6.39) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 13 Nov 2010 22:30:52 +0000 Received: from mambo.cs.wisc.edu (mambo.cs.wisc.edu [128.105.166.23]) by sandstone.cs.wisc.edu (8.14.1/8.14.1) with ESMTP id oADMUV5r008495 for ; Sat, 13 Nov 2010 16:30:31 -0600 Message-ID: <4CDF1187.5030407@cs.wisc.edu> Date: Sat, 13 Nov 2010 16:30:31 -0600 From: Avrilia Floratou User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: user@hive.apache.org Subject: Convert data to BytesRefArrayWritable Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Hi, I want to convert data stored in a hadoop sequence file to BytesRefArrayWritable so that I can use RCFileOutputFormat and create an RCFile. My data contains integers,strings and hashmaps. I guess I don't have to write my own serializer/deserializer for these. I tried using the ColumnarSerDe serializer. It serializes data that are stored in a struct. Should I use the ColumnarStruct to store the data? If yes, then how can I store each row from my dataset to this data structure? I don't see any methods to set the fields. Should I use a different way to get the BytesRefArrayWritable that I need? Thank you, Avrilia