Return-Path: X-Original-To: apmail-asterixdb-notifications-archive@minotaur.apache.org Delivered-To: apmail-asterixdb-notifications-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 769E31807C for ; Fri, 23 Oct 2015 00:35:31 +0000 (UTC) Received: (qmail 45089 invoked by uid 500); 23 Oct 2015 00:35:31 -0000 Delivered-To: apmail-asterixdb-notifications-archive@asterixdb.apache.org Received: (qmail 45056 invoked by uid 500); 23 Oct 2015 00:35:31 -0000 Mailing-List: contact notifications-help@asterixdb.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@asterixdb.incubator.apache.org Delivered-To: mailing list notifications@asterixdb.incubator.apache.org Received: (qmail 45047 invoked by uid 99); 23 Oct 2015 00:35:31 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Oct 2015 00:35:31 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id C0017C6175 for ; Fri, 23 Oct 2015 00:35:30 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.019 X-Spam-Level: X-Spam-Status: No, score=-4.019 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=0.001] autolearn=disabled Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id GlywPVzz3jy1 for ; Fri, 23 Oct 2015 00:35:29 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with SMTP id DB3F9280AB for ; Fri, 23 Oct 2015 00:35:28 +0000 (UTC) Received: (qmail 43812 invoked by uid 99); 23 Oct 2015 00:35:27 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Oct 2015 00:35:27 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id C60322C1F65 for ; Fri, 23 Oct 2015 00:35:27 +0000 (UTC) Date: Fri, 23 Oct 2015 00:35:27 +0000 (UTC) From: "Taewoo Kim (JIRA)" To: notifications@asterixdb.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (ASTERIXDB-1152) IndexLifecycleManager returns wrong index because of resourceID fetching issues MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Taewoo Kim created ASTERIXDB-1152: ------------------------------------- Summary: IndexLifecycleManager returns wrong index because of resourceID fetching issues Key: ASTERIXDB-1152 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1152 Project: Apache AsterixDB Issue Type: Bug Reporter: Taewoo Kim When you have multiple secondary indexes on a dataset and tries to bulkload data into that dataset, an exception happens. The reason that Young-Seok and I observed was one issue. That is, IndexLifecycleManager returns wrong index because of resourceID fetching issues. Specifically, if an index file already exists on disk, it is cached as a local resource. When indexDataFlowHelper tries to get the index by dataset and resource name, IndexLifecyclyManager returns a wrong resource ID to the currently existing index (e.g., LSMBTree index will be returned for LSMInvertedIndexDataflowHelper). You can reproduce this issue by executing one ExecutionTest that creates multiple secondary indexes and does a bulkload - index-leftouterjoin/probe-pidx-with-join-btree-sidx1. Then, you can see the following exception, which is the result of mismatch between the expected index-type and actual index. org.apache.hyracks.api.exceptions.HyracksDataException: java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.hyracks.dataflow.std.sort.ExternalSortRunMerger.process(ExternalSortRunMerger.java:215) at org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:194) at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:85) at org.apache.hyracks.control.nc.Task.run(Task.java:255) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.hyracks.storage.am.common.tuples.TypeAwareTupleWriter.getFieldSlotsBytes(TypeAwareTupleWriter.java:126) at org.apache.hyracks.storage.am.common.tuples.TypeAwareTupleWriter.bytesRequired(TypeAwareTupleWriter.java:40) at org.apache.hyracks.storage.am.lsm.btree.tuples.LSMBTreeTupleWriter.bytesRequired(LSMBTreeTupleWriter.java:43) at org.apache.hyracks.storage.am.btree.frames.BTreeNSMLeafFrame.getBytesRequriedToWriteTuple(BTreeNSMLeafFrame.java:54) at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeBulkLoader.add(BTree.java:974) at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree$LSMBTreeBulkLoader.add(LSMBTree.java:690) at org.apache.hyracks.storage.am.common.dataflow.IndexBulkLoadOperatorNodePushable.nextFrame(IndexBulkLoadOperatorNodePushable.java:95) at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.flush(AbstractFrameAppender.java:83) at org.apache.hyracks.dataflow.std.sort.AbstractFrameSorter.flush(AbstractFrameSorter.java:176) at org.apache.hyracks.dataflow.std.sort.ExternalSortRunMerger.process(ExternalSortRunMerger.java:128) ... 6 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)