Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D3D52200D2D for ; Thu, 12 Oct 2017 15:29:05 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id D26241609CD; Thu, 12 Oct 2017 13:29:05 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 2317E1609E8 for ; Thu, 12 Oct 2017 15:29:04 +0200 (CEST) Received: (qmail 72483 invoked by uid 500); 12 Oct 2017 13:29:04 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 72474 invoked by uid 99); 12 Oct 2017 13:29:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Oct 2017 13:29:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 9E62C1A1314 for ; Thu, 12 Oct 2017 13:29:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id 0id_SaYsH5Au for ; Thu, 12 Oct 2017 13:29:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 3881E60E46 for ; Thu, 12 Oct 2017 13:29:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 7B544E0DE3 for ; Thu, 12 Oct 2017 13:29:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 6BC6E23F36 for ; Thu, 12 Oct 2017 13:29:00 +0000 (UTC) Date: Thu, 12 Oct 2017 13:29:00 +0000 (UTC) From: "Teddy Choi (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-12631) LLAP: support ORC ACID tables MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 12 Oct 2017 13:29:06 -0000 [ https://issues.apache.org/jira/browse/HIVE-12631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-12631: ------------------------------ Attachment: HIVE-12631.29.patch > LLAP: support ORC ACID tables > ----------------------------- > > Key: HIVE-12631 > URL: https://issues.apache.org/jira/browse/HIVE-12631 > Project: Hive > Issue Type: Bug > Components: llap, Transactions > Reporter: Sergey Shelukhin > Assignee: Teddy Choi > Attachments: HIVE-12631.1.patch, HIVE-12631.10.patch, HIVE-12631.10.patch, HIVE-12631.11.patch, HIVE-12631.11.patch, HIVE-12631.12.patch, HIVE-12631.13.patch, HIVE-12631.15.patch, HIVE-12631.16.patch, HIVE-12631.17.patch, HIVE-12631.18.patch, HIVE-12631.19.patch, HIVE-12631.2.patch, HIVE-12631.20.patch, HIVE-12631.21.patch, HIVE-12631.22.patch, HIVE-12631.23.patch, HIVE-12631.24.patch, HIVE-12631.25.patch, HIVE-12631.26.patch, HIVE-12631.27.patch, HIVE-12631.28.patch, HIVE-12631.29.patch, HIVE-12631.3.patch, HIVE-12631.4.patch, HIVE-12631.5.patch, HIVE-12631.6.patch, HIVE-12631.7.patch, HIVE-12631.8.patch, HIVE-12631.8.patch, HIVE-12631.9.patch > > > LLAP uses a completely separate read path in ORC to allow for caching and parallelization of reads and processing. This path does not support ACID. As far as I remember ACID logic is embedded inside ORC format; we need to refactor it to be on top of some interface, if practical; or just port it to LLAP read path. > Another consideration is how the logic will work with cache. The cache is currently low-level (CB-level in ORC), so we could just use it to read bases and deltas (deltas should be cached with higher priority) and merge as usual. We could also cache merged representation in future. -- This message was sent by Atlassian JIRA (v6.4.14#64029)