Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D8226100AB for ; Mon, 8 Apr 2013 23:48:54 +0000 (UTC) Received: (qmail 42532 invoked by uid 500); 8 Apr 2013 23:48:49 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 42432 invoked by uid 500); 8 Apr 2013 23:48:49 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 42424 invoked by uid 99); 8 Apr 2013 23:48:49 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 Apr 2013 23:48:49 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of static.void.dev@gmail.com designates 209.85.220.45 as permitted sender) Received: from [209.85.220.45] (HELO mail-pa0-f45.google.com) (209.85.220.45) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 Apr 2013 23:48:42 +0000 Received: by mail-pa0-f45.google.com with SMTP id kl13so3533800pab.4 for ; Mon, 08 Apr 2013 16:48:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:from:content-type:content-transfer-encoding:subject :message-id:date:to:mime-version:x-mailer; bh=PXGMnkEL1HTCQDrsDN0GbsozRu2oQjDp3vtoeBp2eww=; b=cX6ZBXWFWww5Ezu0k2lymcAByI5rMgg11nexxVLPDIeKZPoJKDeKjJFldQUKqdwixV SS28AcFo/ov+PbrYuKLtJZEvsl1tg998TnGbYqQfM+p+9m4cLFhpYGoPGwh8vOe185T7 dP1pcbuCL6sg9/JdQww7XUe7s2AD58b1lgd5RJAt2WjGLEYeWVsLv0QfNmZGgM7GjVT0 yfA/SiCBg9P+04AK7NBJqPRkf/2egA26mHZZSu0aUa7lYHz1p70VahJ8Ntr/u6o7Op5H 8SuE89sSJ/v48xukD+M4M/MKqpfQDoFd/0xYRp4B6KLu/wjL5PUcBLc4bZ3kpxRYO9hI s03w== X-Received: by 10.67.14.105 with SMTP id ff9mr41943550pad.101.1365464900984; Mon, 08 Apr 2013 16:48:20 -0700 (PDT) Received: from [172.16.1.74] (206-15-64-66.static.twtelecom.net. [206.15.64.66]) by mx.google.com with ESMTPS id ef3sm43451119pad.20.2013.04.08.16.48.18 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 08 Apr 2013 16:48:19 -0700 (PDT) From: Mark Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Best format to use Message-Id: <3F80E783-79EE-4D10-8E84-C5881B9D56EF@gmail.com> Date: Mon, 8 Apr 2013 16:48:17 -0700 To: user@hadoop.apache.org Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on apache.org Trying to determine what the best format to use for storing daily logs. = We recently switch from snappy (.snappy) to gzip (.deflate) but I'm = wondering if there is something better? Our main clients for these daily = logs are pig and hive using an external table. We were thinking about = testing out impala but we see that it doesn't work with compressed text = files. Any suggestions?=20 Thanks=