Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
Received-SPF: pass (nike.apache.org: domain of aaron.sun82@gmail.com
 designates 209.85.214.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <E5C6ED175FFCE34D974527016DF712FD6A9074864F@AMRXM3113.dir.svc.accenture.com>
References: <4E14DDC9.9050504@hotmail.com>
	<BLU0-SMTP1064730F24FFABF48846E8CC2410@phx.gbl>
	<CAPP5OR_2AKuu73L+1mSVOLvqUYOsZA23GPN-3pxgzMvhcFjUXw@mail.gmail.com>
	<BLU0-SMTP13587CF86476BD3F190438CC2410@phx.gbl>
	<CAPbxxbNP76roGUWiTOvH1+Yp=pY728_wSC93bH4jYNVrbfNNPw@mail.gmail.com>
	<E5C6ED175FFCE34D974527016DF712FD6A9074864F@AMRXM3113.dir.svc.accenture.com>
Date: Thu, 7 Jul 2011 18:17:32 -0700
Message-ID: 
 <CALdKvcRv5+y27ZYg2MbcF3duPAYhVe0_noY+Vnr9N8ivvONENw@mail.gmail.com>
Subject: Re: Lzo compression on Hive table
From: Aaron Sun <aaron.sun82@gmail.com>
To: user@hive.apache.org
Content-Type: multipart/alternative; boundary=90e6ba6e865613b81804a78499df

--90e6ba6e865613b81804a78499df
Content-Type: text/plain; charset=UTF-8

You can use this one

STORED AS INPUTFORMAT "com.hadoop.mapred.DeprecatedLzoTextInputFormat"
OUTPUTFORMAT "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"


On Thu, Jul 7, 2011 at 6:06 PM, <jonathan.hwang@accenture.com> wrote:

> Hi there,
> I got my hadoop all setup writing out sequence file with LZO compression.
>  Using the following:
> mapred.output.compress=true
> mapred.output.compression.codec=com.hadoop.compression.lzo.LzoCodec
>
> How do I define my table so it will write out compressed data and be able
> to read in compressed data during my Hive queries?
>
> CREATE EXTERNAL TABLE foo (
>                       columnA string,
>                       columnB string )
>       PARTITIONED BY (date string)
>       ROW FORMAT DELIMITED FIELDS TERMINATED BY "\t"
>       LOCATION '/path/to/hive/tables/foo';
>
> Thanks,
> Jon
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise private information.  If you have
> received it in error, please notify the sender immediately and delete the
> original.  Any other use of the email by you is prohibited.
>

--90e6ba6e865613b81804a78499df
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div>You can use this one</div>
<div>=C2=A0</div>
<div>STORED AS INPUTFORMAT &quot;com.hadoop.mapred.DeprecatedLzoTextInputFo=
rmat&quot;<br>OUTPUTFORMAT &quot;org.apache.hadoop.hive.ql.io.HiveIgnoreKey=
TextOutputFormat&quot;<br><br><br></div>
<div class=3D"gmail_quote">On Thu, Jul 7, 2011 at 6:06 PM, <span dir=3D"ltr=
">&lt;<a href=3D"mailto:jonathan.hwang@accenture.com">jonathan.hwang@accent=
ure.com</a>&gt;</span> wrote:<br>
<blockquote style=3D"BORDER-LEFT: #ccc 1px solid; MARGIN: 0px 0px 0px 0.8ex=
; PADDING-LEFT: 1ex" class=3D"gmail_quote">Hi there,<br>I got my hadoop all=
 setup writing out sequence file with LZO compression. =C2=A0Using the foll=
owing:<br>
mapred.output.compress=3Dtrue<br>mapred.output.compression.codec=3Dcom.hado=
op.compression.lzo.LzoCodec<br><br>How do I define my table so it will writ=
e out compressed data and be able to read in compressed data during my Hive=
 queries?<br>
<br>CREATE EXTERNAL TABLE foo (<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 columnA string,<br>=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 columnB string =
)<br>=C2=A0 =C2=A0 =C2=A0 PARTITIONED BY (date string)<br>=C2=A0 =C2=A0 =C2=
=A0 ROW FORMAT DELIMITED FIELDS TERMINATED BY &quot;\t&quot;<br>
=C2=A0 =C2=A0 =C2=A0 LOCATION &#39;/path/to/hive/tables/foo&#39;;<br><br>Th=
anks,<br>Jon<br><br>This message is for the designated recipient only and m=
ay contain privileged, proprietary, or otherwise private information. =C2=
=A0If you have received it in error, please notify the sender immediately a=
nd delete the original. =C2=A0Any other use of the email by you is prohibit=
ed.<br>
</blockquote></div><br>

--90e6ba6e865613b81804a78499df--