Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of static.void.dev@gmail.com
 designates 209.85.192.180 as permitted sender)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\))
Subject: Re: Best format to use
From: Mark <static.void.dev@gmail.com>
In-Reply-To: <09674432-2891-4729-979A-3E41D0A7F361@gmail.com>
Date: Tue, 9 Apr 2013 09:51:47 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <5FD6A36D-4F5D-4ECB-83A2-CFBFFB321FA7@gmail.com>
References: <3F80E783-79EE-4D10-8E84-C5881B9D56EF@gmail.com>
 <CAOcnVr22dUyR+KGW1XzNFMBm+3VomdPQf6X=w2oyGFHSdE41Ng@mail.gmail.com>
 <09674432-2891-4729-979A-3E41D0A7F361@gmail.com>
To: user@hadoop.apache.org

Actually, compressed sequence files may not work with Pig or Hive then =
right?

On Apr 9, 2013, at 9:50 AM, Mark <static.void.dev@gmail.com> wrote:

> Forgetting Impala, what format would be best to use with daily logs?=20=

>=20
> Block-compressed sequence files?
>=20
> On Apr 8, 2013, at 8:12 PM, Harsh J <harsh@cloudera.com> wrote:
>=20
>> Hey Mark,
>>=20
>> Gzip codec creates extension .gzip, not .deflate (which is
>> DeflateCodec). You may want to re-check your settings.
>>=20
>> Impala questions are best resolved at its current user and developer
>> community at =
https://groups.google.com/a/cloudera.org/forum/#!forum/impala-user.
>> Impala does currently support LZO (and also Indexed LZO) compressed
>> text files however, so you may want to try that as its splittable
>> (compared to Gzip ones).
>>=20
>> On Tue, Apr 9, 2013 at 5:18 AM, Mark <static.void.dev@gmail.com> =
wrote:
>>> Trying to determine what the best format to use for storing daily =
logs. We recently switch from snappy (.snappy) to gzip (.deflate) but =
I'm wondering if there is something better? Our main clients for these =
daily logs are pig and hive using an external table. We were thinking =
about testing out impala but we see that it doesn't work with compressed =
text files. Any suggestions?
>>>=20
>>> Thanks
>>=20
>>=20
>>=20
>> --=20
>> Harsh J
>=20