Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7FD08115FF for ; Wed, 23 Jul 2014 04:17:40 +0000 (UTC) Received: (qmail 21203 invoked by uid 500); 23 Jul 2014 04:17:40 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 21031 invoked by uid 500); 23 Jul 2014 04:17:40 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 20626 invoked by uid 500); 23 Jul 2014 04:17:39 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 20610 invoked by uid 99); 23 Jul 2014 04:17:39 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Jul 2014 04:17:39 +0000 Date: Wed, 23 Jul 2014 04:17:39 +0000 (UTC) From: "Navis (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-4788) RCFile and bzip2 compression not working MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-4788?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-4788: ------------------------ Assignee: Navis Status: Patch Available (was: Open) > RCFile and bzip2 compression not working > ---------------------------------------- > > Key: HIVE-4788 > URL: https://issues.apache.org/jira/browse/HIVE-4788 > Project: Hive > Issue Type: Bug > Components: Compression > Affects Versions: 0.10.0 > Environment: CDH4.2 > Reporter: Johndee Burks > Assignee: Navis > Priority: Minor > Attachments: HIVE-4788.1.patch.txt > > > The issue is that Bzip2 compressed rcfile data is encountering an error w= hen being queried even the most simple query "select *". The issue is easil= y reproducible using the following.=20 > Create a table and load the sample data below.=20 > DDL: create table source_data (a string, b string) row format delimited f= ields terminated by ','; > Sample data:=20 > apple,sauce=20 > Test:=20 > Do the following and you should receive the error listed below for the rc= file table with bz2 compression.=20 > create table rc_nobz2 (a string, b string) stored as rcfile;=20 > insert into table rc_nobz2 select * from source_txt;=20 > SET io.seqfile.compression.type=3DBLOCK;=20 > SET hive.exec.compress.output=3Dtrue;=20 > SET mapred.compress.map.output=3Dtrue;=20 > SET mapred.output.compress=3Dtrue;=20 > SET mapred.output.compression.codec=3Dorg.apache.hadoop.io.compress.BZip2= Codec;=20 > create table rc_bz2 (a string, b string) stored as rcfile;=20 > insert into table rc_bz2 select * from source_txt;=20 > hive> select * from rc_bz2;=20 > Failed with exception java.io.IOException:java.io.IOException: Stream is = not BZip2 formatted: expected 'h' as first byte but got '=EF=BF=BF'=20 > hive> select * from rc_nobz2;=20 > apple=09sauce -- This message was sent by Atlassian JIRA (v6.2#6252)