Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8A9EE200CF0 for ; Thu, 7 Sep 2017 23:07:09 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 890BE160E1C; Thu, 7 Sep 2017 21:07:09 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D79C81609D0 for ; Thu, 7 Sep 2017 23:07:08 +0200 (CEST) Received: (qmail 62099 invoked by uid 500); 7 Sep 2017 21:07:08 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 62082 invoked by uid 99); 7 Sep 2017 21:07:08 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Sep 2017 21:07:08 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 98250CDF99 for ; Thu, 7 Sep 2017 21:07:07 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id PikuoZXhL9at for ; Thu, 7 Sep 2017 21:07:03 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 9EC0961045 for ; Thu, 7 Sep 2017 21:07:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id E9198E0ED9 for ; Thu, 7 Sep 2017 21:07:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 44B2A2415B for ; Thu, 7 Sep 2017 21:07:00 +0000 (UTC) Date: Thu, 7 Sep 2017 21:07:00 +0000 (UTC) From: "Sergey Shelukhin (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-12631) LLAP: support ORC ACID tables MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 07 Sep 2017 21:07:09 -0000 [ https://issues.apache.org/jira/browse/HIVE-12631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-12631: ------------------------------------ Attachment: HIVE-12631.28.patch > LLAP: support ORC ACID tables > ----------------------------- > > Key: HIVE-12631 > URL: https://issues.apache.org/jira/browse/HIVE-12631 > Project: Hive > Issue Type: Bug > Components: llap, Transactions > Reporter: Sergey Shelukhin > Assignee: Teddy Choi > Attachments: HIVE-12631.10.patch, HIVE-12631.10.patch, HIVE-12631.11.patch, HIVE-12631.11.patch, HIVE-12631.12.patch, HIVE-12631.13.patch, HIVE-12631.15.patch, HIVE-12631.16.patch, HIVE-12631.17.patch, HIVE-12631.18.patch, HIVE-12631.19.patch, HIVE-12631.1.patch, HIVE-12631.20.patch, HIVE-12631.21.patch, HIVE-12631.22.patch, HIVE-12631.23.patch, HIVE-12631.24.patch, HIVE-12631.25.patch, HIVE-12631.26.patch, HIVE-12631.27.patch, HIVE-12631.28.patch, HIVE-12631.2.patch, HIVE-12631.3.patch, HIVE-12631.4.patch, HIVE-12631.5.patch, HIVE-12631.6.patch, HIVE-12631.7.patch, HIVE-12631.8.patch, HIVE-12631.8.patch, HIVE-12631.9.patch > > > LLAP uses a completely separate read path in ORC to allow for caching and parallelization of reads and processing. This path does not support ACID. As far as I remember ACID logic is embedded inside ORC format; we need to refactor it to be on top of some interface, if practical; or just port it to LLAP read path. > Another consideration is how the logic will work with cache. The cache is currently low-level (CB-level in ORC), so we could just use it to read bases and deltas (deltas should be cached with higher priority) and merge as usual. We could also cache merged representation in future. -- This message was sent by Atlassian JIRA (v6.4.14#64029)