From issues-return-4708-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Wed Feb 20 03:13:05 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 3502918077A for ; Wed, 20 Feb 2019 04:13:05 +0100 (CET) Received: (qmail 80779 invoked by uid 500); 20 Feb 2019 03:13:04 -0000 Mailing-List: contact issues-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list issues@phoenix.apache.org Received: (qmail 80765 invoked by uid 99); 20 Feb 2019 03:13:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Feb 2019 03:13:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 96DCD189EED for ; Wed, 20 Feb 2019 03:13:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id Deg01lq_9FjJ for ; Wed, 20 Feb 2019 03:13:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 9D143623FF for ; Wed, 20 Feb 2019 03:13:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id E6E6FE00A9 for ; Wed, 20 Feb 2019 03:13:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id A261524517 for ; Wed, 20 Feb 2019 03:13:00 +0000 (UTC) Date: Wed, 20 Feb 2019 03:13:00 +0000 (UTC) From: "Hudson (JIRA)" To: issues@phoenix.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (PHOENIX-5018) Index mutations created by UPSERT SELECT will have wrong timestamps MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PHOENIX-5018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772583#comment-16772583 ] Hudson commented on PHOENIX-5018: --------------------------------- FAILURE: Integrated in Jenkins build Phoenix-4.x-HBase-1.3 #334 (See [https://builds.apache.org/job/Phoenix-4.x-HBase-1.3/334/]) PHOENIX-5018 Index mutations created by UPSERT SELECT will have wrong (gjacoby: rev 6c25ec031d2ee198e628e014988c8ca7f999146a) * (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/IndexBuildTimestampIT.java * (edit) phoenix-core/src/it/java/org/apache/phoenix/rpc/PhoenixServerRpcIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/mapreduce/PhoenixRecordReader.java * (add) phoenix-core/src/main/java/org/apache/phoenix/compile/ServerBuildIndexCompiler.java * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/IndexToolIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/mapreduce/util/PhoenixConfigurationUtil.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/mapreduce/util/PhoenixMapReduceUtil.java * (add) phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/PhoenixServerBuildIndexMapper.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/mapreduce/PhoenixInputFormat.java * (add) phoenix-core/src/main/java/org/apache/phoenix/mapreduce/PhoenixServerBuildIndexInputFormat.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexTool.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/TableDDLPermissionsIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java > Index mutations created by UPSERT SELECT will have wrong timestamps > ------------------------------------------------------------------- > > Key: PHOENIX-5018 > URL: https://issues.apache.org/jira/browse/PHOENIX-5018 > Project: Phoenix > Issue Type: Bug > Affects Versions: 4.14.0, 5.0.0 > Reporter: Geoffrey Jacoby > Assignee: Kadir OZDEMIR > Priority: Major > Fix For: 4.15.0, 5.1 > > Attachments: PHOENIX-5018.4.x-HBase-1.3.001.patch, PHOENIX-5018.4.x-HBase-1.3.002.patch, PHOENIX-5018.4.x-HBase-1.4.001.patch, PHOENIX-5018.4.x-HBase-1.4.002.patch, PHOENIX-5018.master.001.patch, PHOENIX-5018.master.002.patch, PHOENIX-5018.master.003.patch, PHOENIX-5018.master.004.patch > > Time Spent: 5.5h > Remaining Estimate: 0h > > When doing a full rebuild (or initial async build) of a local or global index using IndexTool and PhoenixIndexImportDirectMapper, or doing a synchronous initial build of a global index using the index create DDL, we generate the index mutations by using an UPSERT SELECT query from the base table to the index. > The timestamps of the mutations use the default HBase behavior, which is to take the current wall clock. However, the timestamp of an index KeyValue should use the timestamp of the initial KeyValue in the base table. > Having base table and index timestamps out of sync can cause all sorts of weird side effects, such as if the base table has data with an expired TTL that isn't expired in the index yet. Also inserting old mutations with new timestamps may overwrite the data that has been newly overwritten by the regular data path during index build, which would lead to data loss and inconsistency issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)