Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 91FA5102C0 for ; Tue, 21 Jan 2014 05:16:01 +0000 (UTC) Received: (qmail 95142 invoked by uid 500); 21 Jan 2014 05:15:58 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 94862 invoked by uid 500); 21 Jan 2014 05:15:50 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 94755 invoked by uid 99); 21 Jan 2014 05:15:47 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 21 Jan 2014 05:15:47 +0000 X-ASF-Spam-Status: No, hits=-5.0 required=5.0 tests=RCVD_IN_DNSWL_HI,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of Chandra.Bogala@gs.com designates 204.4.187.100 as permitted sender) Received: from [204.4.187.100] (HELO mxecd01.gs.com) (204.4.187.100) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 21 Jan 2014 05:15:41 +0000 X-IronPort-AV: E=Sophos;i="4.95,694,1384318800"; d="scan'208";a="44690633" Received: from unknown (HELO mxpbd01-public.ny.fw.gs.com) ([148.86.115.129]) by mxecd01.idz.gs.com with ESMTP; 21 Jan 2014 00:15:20 -0500 From: "Bogala, Chandra Reddy" X-sendergroup: RELAYLIST Received: from gshcbdp12ex.firmwide.corp.gs.com ([10.135.172.21]) by mxpbd01.ny.fw.gs.com with ESMTP; 21 Jan 2014 00:15:20 -0500 Received: from GSCMBLP12EX.firmwide.corp.gs.com ([139.172.99.7]) by gshcbdp12ex.firmwide.corp.gs.com ([10.135.172.21]) with mapi; Tue, 21 Jan 2014 00:15:20 -0500 To: "'user@hive.apache.org'" Date: Tue, 21 Jan 2014 00:15:16 -0500 Subject: RE: casting complex data types for outputs of custom scripts Thread-Topic: casting complex data types for outputs of custom scripts Thread-Index: Ac8VeAHR27E4n50ATRWQGfhAgqlLFAA73nvg Message-ID: <0A88212BF9DDF54FBEE61A2C1965291D0324043D5B@GSCMBLP12EX.firmwide.corp.gs.com> References:

In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US x-retentionstamp: Firmwide Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org Can it be possible to share python script which does the conversion? Thanks, Chandra -----Original Message----- From: rohan monga [mailto:monga.rohan@gmail.com]=20 Sent: Monday, January 20, 2014 6:08 AM To: user@hive.apache.org Subject: Re: casting complex data types for outputs of custom scripts sorry for the delayed response. yes the python script follows that. -- Rohan Monga On Tue, Jan 14, 2014 at 4:31 PM, Stephen Sprague wrote= : > @OP - first thing i'd ask is does your python script obey the=20 > ^A,^B,^C,^D etc. nesting delimiter pattern. give that your create=20 > table does not specify delimiters those are the defaults. nb. ^A =3D=3D= =20 > control-A =3D=3D \001 > > Cheers, > Stephen. > > > On Tue, Jan 14, 2014 at 3:11 PM, Andre Araujo wrote: >> >> I had a similar issue in the past when trying to cast an empty array=20 >> to array(). By default Hive assumes it's an array(). >> I don't think there's currently a Hive syntax to cast values to=20 >> complex data types. If there's one, I'd love to know what it is :) >> >> >> On 14 January 2014 10:22, rohan monga wrote: >>> >>> Hi, >>> >>> I have a table that is of the following format >>> >>> create table t1 ( f1 int, f2 array> ); >>> >>> Now I have a custom script that does some computation and generates=20 >>> the value for f2 like so >>> >>> from ( >>> from randomtable r >>> map r.g1, r.g2, r.g3 >>> using '/bin/cat' as g1, g2, g3 >>> cluster by g1 ) m >>> insert overwrite table t1 >>> reduce m.g1, m.g2, m.g3 >>> using 'python customScript.py' as ( f1 , f2 ); >>> >>> however f2 is not being loaded properly into t1, it comes up broken=20 >>> or null. What should I do so that f2 is loaded as an array of structs. >>> >>> >>> Thanks, >>> >>> -- >>> Rohan Monga >> >> >> >> >> -- >> Andr=E9 Ara=FAjo >> Big Data Consultant/Solutions Architect The Pythian Group - Australia=20 >> - www.pythian.com >> >> Office (calls from within Australia): 1300 366 021 x1270 >> Office (international): +61 2 8016 7000 x270 OR +1 613 565 8696 x1270 >> Mobile: +61 410 323 559 >> Fax: +61 2 9805 0544 >> IM: pythianaraujo @ AIM/MSN/Y! or araujo@pythian.com @ GTalk >> >> "Success is not about standing at the top, it's the steps you leave=20 >> behind." - Iker Pou (rock climber) >> >> -- >> >> >> >