Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E2602119D0 for ; Fri, 20 Jun 2014 15:54:07 +0000 (UTC) Received: (qmail 27454 invoked by uid 500); 20 Jun 2014 15:54:06 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 27381 invoked by uid 500); 20 Jun 2014 15:54:06 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 27371 invoked by uid 99); 20 Jun 2014 15:54:06 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Jun 2014 15:54:06 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy includes SPF record at spf.trusted-forwarder.org) Received: from [209.85.192.43] (HELO mail-qg0-f43.google.com) (209.85.192.43) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Jun 2014 15:54:04 +0000 Received: by mail-qg0-f43.google.com with SMTP id z60so3569468qgd.2 for ; Fri, 20 Jun 2014 08:53:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=Cs6CWocXT0pPTPtsyhw4TTWNX3g/hiDZ1ereWDIEkHM=; b=RInk31SgvQboyvSdWdw8acyt34QHnkNzthySH3AUTYDddgKJa5kAd7Vhi9o+FRhWk9 Ea54+wbG3AJSaaJg1Ht2vkDs6UVQnxSrimrH+Cca38HlX3DlSJFt3Ef1HBIJXMr21vPF o3GmVt+S/DF+BONAMaUqRR4rhhsCo9I2wEuR2Q6vqTZr8pdx5yHjl9QhQMQrDMUleMla qnYhgE5GK/COsBSewpfZJjXZ8MmpfRM/QAGWCJ/oR0ZVtShDAtuLSasFJGHFoXZzKr6O txbV0P7HQ/V8fItPEt0kwcOP7WPKoYRoXC4p8lKj0LW/9VGZgYPbO4g55cegLWALpx51 KPIg== X-Gm-Message-State: ALoCoQmdN4nstI9IKf0SS/Jwl1jIciDU2ALVxC13AoTPix/8bFxdBY1JZhhvC8a889dKqb5FCbHu MIME-Version: 1.0 X-Received: by 10.229.127.9 with SMTP id e9mr6268219qcs.5.1403279620043; Fri, 20 Jun 2014 08:53:40 -0700 (PDT) Received: by 10.140.31.247 with HTTP; Fri, 20 Jun 2014 08:53:39 -0700 (PDT) In-Reply-To: References: Date: Fri, 20 Jun 2014 17:53:39 +0200 Message-ID: Subject: Re: How to load json data with nested arrays into hive? From: Christian Link To: user@hive.apache.org Content-Type: multipart/alternative; boundary=001a113334aa460a5804fc467fd6 X-Virus-Checked: Checked by ClamAV on apache.org --001a113334aa460a5804fc467fd6 Content-Type: text/plain; charset=UTF-8 aha! ok...let me try this hmmm... I just run a master and two core nodes...I guess the jar needs to be put somewhere on the core nodes? Thanks! On Fri, Jun 20, 2014 at 5:37 PM, Edward Capriolo wrote: > I've also added the jar to my hive session with ADD JAR... > > This is not correct. For input formats/serde the file must be in your > auxlib > > mkdir $HIVE_HOME/auxlib > cp $HIVE_HOME/auxlib > > Really good json support > https://github.com/rcongiu/Hive-JSON-Serde > > > On Fri, Jun 20, 2014 at 11:35 AM, Christian Link > wrote: > >> Hey Douglas, >> >> thanks a lot. >> That's what I've tried - but it didn't work for me. >> >> I've created table with ROW FORMAT "one of the SerDe" and an array which >> looked like the array in the json file. >> >> I've also added the jar to my hive session with ADD JAR... >> >> Loading data into that table with the array failed. >> >> I'm totally new to this and I guess I'm doing wrong right a the start >> (loading data into a hive table or so)... >> >> Best, >> Chris >> >> >> On Fri, Jun 20, 2014 at 5:29 PM, Douglas Moore < >> douglas.moore@thinkbiganalytics.com> wrote: >> >>> Hi Chris, >>> >>> Try this Serde , I looked >>> at 6 of them a while back and this one was the best, including dealing with >>> nested JSON structures: >>> https://github.com/rcongiu/Hive-JSON-Serde >>> >>> - Douglas >>> >>> >> > --001a113334aa460a5804fc467fd6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
aha!
ok...let me try this

hmm= m... I just run a master and two core nodes...I guess the jar needs to be p= ut somewhere on the core nodes?

Thanks!


On Fri, Jun 2= 0, 2014 at 5:37 PM, Edward Capriolo <edlinuxguru@gmail.com> wrote:
<= div>I've also added the jar to my hive session with ADD JAR...

<= /div>
This is not correct. For input formats/serde the file must be in your= auxlib

mkdir $HIVE_HOME/auxlib
cp <whatever.jar> $HIVE_HOME/auxlib

Really good json sup= port
https://github.com/rcongiu/Hive-JSON-Serde


On Fri, Jun 20, 2014 at 11:35 AM, Christian = Link <christian.link@mdmp.com> wrote:
Hey Douglas,

thanks a lot.
Th= at's what I've tried - but it didn't work for me.
I've created table with ROW FORMAT "one of the SerDe&q= uot; and an array which looked like the array in the json file.

I've also added the jar to my hive session with ADD= JAR...

Loading data into that table with the arra= y failed.

I'm totally new to this and I guess = I'm doing wrong right a the start (loading data into a hive table or so= )...

Best,
Chris


On Fri, Jun 20, 2014 at 5:29 PM, Do= uglas Moore <douglas.moore@thinkbiganalytics.com>= wrote:
Hi Chris,

Try = this Serde, I looked at 6 of them a while back and this one was the best,= including dealing with nested JSON structures:

- Douglas
<= br>



--001a113334aa460a5804fc467fd6--