Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9BBD217EB0 for ; Fri, 3 Apr 2015 23:01:10 +0000 (UTC) Received: (qmail 28061 invoked by uid 500); 3 Apr 2015 23:01:09 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 27992 invoked by uid 500); 3 Apr 2015 23:01:09 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 27976 invoked by uid 99); 3 Apr 2015 23:01:09 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Apr 2015 23:01:09 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of thejas.nair@gmail.com designates 209.85.213.169 as permitted sender) Received: from [209.85.213.169] (HELO mail-ig0-f169.google.com) (209.85.213.169) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Apr 2015 23:01:03 +0000 Received: by ignm3 with SMTP id m3so71650976ign.0 for ; Fri, 03 Apr 2015 15:59:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=2twazZ99+fSpxLXEnPEvJyjhxBmF0rYH5qf0/9KJenI=; b=i+Zr7Zuly2vuY4yC7Qcr4WLrvs9gG5zyu32zUSglAhw6E2P2rh12D0nP/PJ77uS6b5 L8oze+GPR7WGZk8McoRW71tLJVEA8AU8AgA22uKXZrdORenpjM6S4MwlwYeafYELF+TG UhFiU5ATnCzvof5pNgmfJMoqQsvFZtJdSbT75CRNGE03YIWhnpEkblnRF/hewV+GvgR+ JKM9yHAVzV35srfxendNn/QLQtgH1ogQRMgwyQ7C3DG0L9JYXVivIY04uKUaDDrO05kD XA8R0O20XjhpoX6GFDMpkYyWdQp1jpYXKQ7IsQKY1KIrSWN2tdBNNam9VL4k1DuJyEM2 wAzw== MIME-Version: 1.0 X-Received: by 10.50.111.168 with SMTP id ij8mr7880761igb.43.1428101953320; Fri, 03 Apr 2015 15:59:13 -0700 (PDT) Received: by 10.36.104.133 with HTTP; Fri, 3 Apr 2015 15:59:13 -0700 (PDT) In-Reply-To: References: <551C2690.6050201@gmail.com> <551EEF29.7050707@gmail.com> Date: Fri, 3 Apr 2015 15:59:13 -0700 Message-ID: Subject: Re: ORC separate project From: Thejas Nair To: dev Content-Type: multipart/alternative; boundary=047d7b414534a17e5f0512d9e53e X-Virus-Checked: Checked by ClamAV on apache.org --047d7b414534a17e5f0512d9e53e Content-Type: text/plain; charset=UTF-8 On Fri, Apr 3, 2015 at 1:25 PM, Lefty Leverenz wrote: > Hive users who wished to use ORC would obviously need to pull in ORC >> artifacts in addition to Hive. >> > > What would happen with Hive features that (currently) only work with ORC? > Would they be extended to work with other file formats and stay in Hive? > What about future features -- would they have to work with multiple file > formats from the get-go? > The storage-api module proposed above would lead to clearer storage interfaces in hive. That will in turn help to implement such features using other storage including parquet, hbase etc. The result of this work will not automatically make those features worth with ORC, somebody would need to do that. Whether future features would work for all formats would depend on whether the new feature needs new functionality to be supported by the storage layer. If the feature needs new storage functionality, I would expect new interfaces to be defined in hive, and then implemented by the storage engines that want to support that feature. This will not negatively impact experience of users with respect to ORC or other storage formats. The way we package parquet in hive, we can package ORC as well. In fact, users would be more easily be able to upgrade their version of ORC being used, as releases can happen independent of each other. --047d7b414534a17e5f0512d9e53e--