Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C16611869C for ; Fri, 23 Oct 2015 21:21:07 +0000 (UTC) Received: (qmail 72718 invoked by uid 500); 23 Oct 2015 21:21:06 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 72646 invoked by uid 500); 23 Oct 2015 21:21:06 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 72632 invoked by uid 99); 23 Oct 2015 21:21:06 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Oct 2015 21:21:06 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id BD14AC17B9 for ; Fri, 23 Oct 2015 21:21:05 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.121 X-Spam-Level: X-Spam-Status: No, score=-0.121 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id hNeyW6HpaePv for ; Fri, 23 Oct 2015 21:21:05 +0000 (UTC) Received: from mail-ig0-f180.google.com (mail-ig0-f180.google.com [209.85.213.180]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id 7729B20751 for ; Fri, 23 Oct 2015 21:21:04 +0000 (UTC) Received: by igbhv6 with SMTP id hv6so24245424igb.0 for ; Fri, 23 Oct 2015 14:20:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DHfqP9Nr8RBxxCka1H2VCJ1gfaD4eQrCcFbFgW2j6PE=; b=ZTN5i0dGbjiUoNXX109xk9xmpMtXa1FA0qqci6nyz79JJ8lBxZL5uRc6bOrmLYB2cu cWNDa97mcz6LmfauYtT1rsp4KOMxK6IUfrD+cYCVOul3FuDUGxDT0Shy1ASBcWDMPm7X WRAuOm+XKi5zNqTZpssea7jR9DCCapDkPCpmR9wXyVTi+5ohCdJKmFREbWr3ocdNvGzn I/V7/rraAQvJkygcyb8dCFnxWFeusbdPCQ504Ar++uR5HH5ca5D5nciAWB3uj6hrKbcl Hz5zXfBCcJp32aqT21nLkz5DQiMHjfLW93wURhuFJuvSkrl0hzjq2Ze9WT1qWgcRASIi AMlA== MIME-Version: 1.0 X-Received: by 10.50.65.8 with SMTP id t8mr7042689igs.74.1445635257183; Fri, 23 Oct 2015 14:20:57 -0700 (PDT) Received: by 10.79.38.5 with HTTP; Fri, 23 Oct 2015 14:20:57 -0700 (PDT) In-Reply-To: References: Date: Fri, 23 Oct 2015 17:20:57 -0400 Message-ID: Subject: Re: Issue with job serialization formats mangling results From: Aaron Wiebe To: Gopal Vijayaraghavan Cc: "user@hive.apache.org" Content-Type: text/plain; charset=UTF-8 Right on - that solved it. Thanks Gopal. On Fri, Oct 23, 2015 at 3:31 PM, Gopal Vijayaraghavan wrote: > > >>I've then created ORC and Parquet versions of this same table. The >>behavior remains... select * works, any filter creates horribly >>mangled results. >> >>To replace- throw this into a file: >> >>{"id":1,"order_id":8,"number":1,"broken":"#\n---\nstuff\nstuff2: >>\"stuff3\"\nstuff4: '730'\nstuff5: []\n","last":null} > > You're trying to fix the issue on the wrong side of the problem, I think. > > Try with > > set > hive.default.serde=org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe > ; > set hive.query.result.fileformat=SequenceFile; > > > Hopefully we'll have a newer & more compact format for results soon. > > Cheers, > Gopal > >