From issues-return-3114-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Mon Dec 3 10:25:07 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 7503D18067A for ; Mon, 3 Dec 2018 10:25:06 +0100 (CET) Received: (qmail 20574 invoked by uid 500); 3 Dec 2018 09:25:05 -0000 Mailing-List: contact issues-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list issues@phoenix.apache.org Received: (qmail 20554 invoked by uid 99); 3 Dec 2018 09:25:05 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Dec 2018 09:25:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id EA865C0413 for ; Mon, 3 Dec 2018 09:25:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id NdOTXDI1n07N for ; Mon, 3 Dec 2018 09:25:03 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id B300E5FAF1 for ; Mon, 3 Dec 2018 09:25:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 848C4E261D for ; Mon, 3 Dec 2018 09:25:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id E384D27772 for ; Mon, 3 Dec 2018 09:25:00 +0000 (UTC) Date: Mon, 3 Dec 2018 09:25:00 +0000 (UTC) From: "Monani Mihir (JIRA)" To: issues@phoenix.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (PHOENIX-5048) Index Rebuilder does not handle INDEX_STATE timestamp check for all index MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PHOENIX-5048?page=3Dcom.atlassi= an.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D16= 706873#comment-16706873 ]=20 Monani Mihir commented on PHOENIX-5048: --------------------------------------- [~vincentpoon] No, I don't think we need to include else part under try-cat= ch block because else block code will be use in case of rebuilding index wi= th IndexTool as Rebuiilder tool might be lagging a lot and we want to use M= R job for Index Rebuild. What do you think?=C2=A0 > Index Rebuilder does not handle INDEX_STATE timestamp check for all index > ------------------------------------------------------------------------- > > Key: PHOENIX-5048 > URL: https://issues.apache.org/jira/browse/PHOENIX-5048 > Project: Phoenix > Issue Type: Improvement > Affects Versions: 4.14.0, 4.14.1 > Reporter: Monani Mihir > Assignee: Monani Mihir > Priority: Major > Attachments: PHOENIX-5048.patch > > > After rebuilder is finished for Partial Index Rebuild, It will check if I= ndex state has been updated after Upper bound of the scan we use in partial= index Rebuild. If that happens then it will fail Index Rebuild as Index wr= ite failure occured while we were rebuilding Index. > {code:java} > MetaDataEndpointImpl.java#updateIndexState() > public void updateIndexState(RpcController controller, UpdateIndexStateRe= quest request, > RpcCallback done) { > ... > // If the index status has been updated after the upper bound of the scan= we use > // to partially rebuild the index, then we need to fail the rebuild becau= se an > // index write failed before the rebuild was complete. > if (actualTimestamp > expectedTimestamp) { > builder.setReturnCode(MetaDataProtos.MutationCode.UNALLOWED_TABLE_MUTATIO= N); > builder.setMutationTime(EnvironmentEdgeManager.currentTimeMillis()); > done.run(builder.build()); > return; > } > ... > }{code} > After Introduction of=C2=A0TrackingParallelWriterIndexCommitter [PHOENIX-= 3815|https://issues.apache.org/jira/browse/PHOENIX-3815], we only disable I= ndex which get failure . Before that , in=C2=A0ParallelWriterIndexCommitter= we were disabling all index even if=C2=A0Index failure happens for=C2=A0on= e Index only.=C2=A0 > Suppose Data Table has 3 index and above condition becomes true for first= index , then we won't even check for remain two Index. > {code:java} > MetaDataRegionObserver.java#BuildIndexScheduleTask.java#run() > for (PTable indexPTable : indexesToPartiallyRebuild) { > String indexTableFullName =3D SchemaUtil.getTableName( > indexPTable.getSchemaName().getString(), > indexPTable.getTableName().getString()); > if (scanEndTime =3D=3D latestUpperBoundTimestamp) { > IndexUtil.updateIndexState(conn, indexTableFullName, PIndexState.ACTIVE, = 0L, latestUpperBoundTimestamp); > batchExecutedPerTableMap.remove(dataPTable.getName()); > LOG.info("Making Index:" + indexPTable.getTableName() + " active after re= building"); > } else { > // Increment timestamp so that client sees updated disable timestamp > IndexUtil.updateIndexState(conn, indexTableFullName, indexPTable.getIndex= State(), scanEndTime * signOfDisableTimeStamp, latestUpperBoundTimestamp); > Long noOfBatches =3D batchExecutedPerTableMap.get(dataPTable.getName()); > if (noOfBatches =3D=3D null) { > noOfBatches =3D 0l; > } > batchExecutedPerTableMap.put(dataPTable.getName(), ++noOfBatches); > LOG.info("During Round-robin build: Successfully updated index disabled t= imestamp for " > + indexTableFullName + " to " + scanEndTime); > } > } > {code} > =C2=A0 > =C2=A0 > =C2=A0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)